amadan
Design: the replicator's authority, revision 5 (the helper's own per-inode lock bookkeeping; the fence shared with every helper and recovery as verified termination; a dead helper's admissions settled by the running agent; closure only with discoveries committed; reach over the unacknowledged ancestry with root-first acknowledgement; the sleeping binding's incarnation and direct freshness; a lineage of filesystem incarnation only; the walk's grammar kept; spike notes superseded where wrong)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2 files changed,
+14
−14
docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md+11 −11docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md+3 −3
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md| index 139cafda8..44688f7ae 100644 |
| --- a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| +++ b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-design.md |
| @@ -1,6 +1,6 @@ |
| # The replicator's authority: retained descriptors, a durable admission, claims by identity |
| -Date: 2026-10-03. Status: design, revision 4, for review (revision 3 was rejected by Astra on five P1s: two SQLite VFSs in one process cannot coordinate through POSIX locks, since the library's own per-inode bookkeeping is in-process; a recorder that abandons early could settle an admission before its transport ended; an admission-only barrier does not cover a discovery whose uncertainty write failed after every admission settled; links at acknowledgement contradicted acknowledging only after settlement, and an inherited uncertainty vanished at the next successor; the ledger's lock does not prove the old process gone; and four P2s: closure at zero pending and the margin's extra day, sleeping claims with no binding, absolute and `.` spellings in the walk, `(dev, ino)` as a lineage; each answered here. Revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| +Date: 2026-10-03. Status: design, revision 5, for review (revision 4 was rejected by Astra on four P1s: the helper has several connections of its own and needs SQLite's within-process lock bookkeeping between them; the producer fence did not cover a helper that outlives the agent with an admission in hand; an intermediate fleet closed before acknowledging broke transitive protection; a fleet could be marked closed with a discovery uncommitted; and four P2s: a dead helper's admissions had no terminal in the running agent, the sleeping binding persisted a reusable inode, the lineage predicate named header fields that identify nothing, the walk's grammar refused a `.` target the predicate accepts; each answered here. Revision 3 was rejected by Astra on five P1s: two SQLite VFSs in one process cannot coordinate through POSIX locks, since the library's own per-inode bookkeeping is in-process; a recorder that abandons early could settle an admission before its transport ended; an admission-only barrier does not cover a discovery whose uncertainty write failed after every admission settled; links at acknowledgement contradicted acknowledging only after settlement, and an inherited uncertainty vanished at the next successor; the ledger's lock does not prove the old process gone; and four P2s: closure at zero pending and the margin's extra day, sleeping claims with no binding, absolute and `.` spellings in the walk, `(dev, ino)` as a lineage; each answered here. Revision 2 was rejected by Astra on three P1s: the VFS delegated SQLite's Unix file methods, which open the SHM and check moved files by pathname themselves; every settled admission wrote a permanent hold, so ordinary replication never read complete; a late owner-scoped extension did not reach successors that had already acknowledged; and six P2s: orphan admissions never terminated, `..` and chain edges unspecified, `dup` sharing the directory offset, the platform-link predicate not preserved and terminal links unclassified, the uid producer without a lifecycle, one billing owner not one replicator; each answered here. Revision 1 was rejected by Astra on five P1s: SQLite canonicalises a `/proc` pathname before it opens anything and this driver passes no `nofollow`; `os.Root` confines but re-resolves every name, so a swap between a listing and an inspection still moves the answer; a `Start` row is not an admission, since the meter sends without one; the attempt-derived window lost a request completed before the change was noticed and the days between an admission and a finish; `closed_at` was not a handover; plus the directory open before `O_DIRECTORY`, per-owner holds the ledger cannot store, a uid with no producer, I3 against Part D, and tests). Branch `design-replicator-pinned-root`, off `feat/carloku-metered-replicator-20261001` at `e1280811`. It answers the eleven findings the twentieth review deferred (`DISPOSITIONS-metered-replicator.md`, review twenty) and keeps decisions 10 to 13 and approval H (`DECISIONS-FOR-PAUL.md`, section D, on the launch branch) as parameters. Nothing is built, wired or applied. |
| ## What stands today |
| @@ -10,7 +10,7 @@ The replicator (`internal/ops/litestream_metered.go`, `litestream_owners.go`; pa |
| **I1, identity through retained descriptors.** (a) *Containment*: no operation of the replicator or of litestream on a tenant's files resolves a pathname above the directory the fleet accepted for that file; (b) *retention*: the descriptor that listed a directory is the descriptor against which its entries are inspected, its links read, their targets classified and its files opened, for that look, so nothing between a listing and an inspection can move the answer; (c) *acceptance*: an owned file is opened by the descriptor of its accepted directory with no link followed in the last component, and the opened file's `(dev, ino)` equals the accepted facts before any use. Retention is authority over directories and checked identities, not a frozen inventory, and it has two roots: the *accepted traversal root* (a platform root or a tenant root) through which acquisition and link classification may traverse retained ancestors, and the *accepted immediate directory* of an owned file, which every data open uses and nothing else. Ancestor replacement cannot redirect an operation, and an identity mismatch at acquisition refuses use; entries inside a retained directory can change, and a swap put back between two looks is not detected by looking; what (a) to (c) promise is that no operation is steered by such a change, and that every change a look does see is held for. Ascent never uses the kernel's `..`: it pops the retained stack. Each acquisition compares what it opened with the facts of the entry it was acquired from, and each later look compares every named edge (parent descriptor, name) with the retained child's facts, so a moved or replaced component is seen as a change at the next look. |
| -**I2, a durable admission, two kinds of hold.** No request is sent until a row naming it (fleet, owner, admission instant) is durable. A *barrier* is temporary: a day is not published complete for an admission's owner while that admission is unsettled, and lifts when it settles. An *uncertainty* is permanent: an interval written when a fleet's authority is found changed, from the earlier of today's lower bound (the registry proof, the last clean look's start) and the earliest admission of the fleet then unsettled, minus the margin, open until every admission of the fleet is settled, then closed at the latest settlement plus the margin; it holds its days for the fleet's owners and for every owner linked to it (below). An admission that settles with no uncertainty open on its fleet writes nothing permanent: the ordinary day of an unchanged fleet becomes complete. Discovery and its first hold are recorded in one transaction; if that transaction fails, a publication-visible *pending discovery* (the ledger's failure latch, extended with the lower bound) bars every day from that bound for every owner the fleet reaches until the write succeeds, and the fleet refuses every request meanwhile. Settlement is the last of the recorder's and the transport's completions, written by whichever lands last, with the upper bound no earlier than both. A fleet's effective uncertainties are its own and every one it inherited, and acknowledgement carries the whole set on. A successor acknowledges a predecessor only when the predecessor is terminal, and the acknowledgement links the successor's owners to every uncertainty of the predecessor still open, so a late extension reaches them. |
| +**I2, a durable admission, two kinds of hold.** No request is sent until a row naming it (fleet, owner, admission instant) is durable. A *barrier* is temporary: a day is not published complete for an admission's owner while that admission is unsettled, and lifts when it settles. An *uncertainty* is permanent: an interval written when a fleet's authority is found changed, from the earlier of today's lower bound (the registry proof, the last clean look's start) and the earliest admission of the fleet then unsettled, minus the margin, open until every admission of the fleet is settled, then closed at the latest settlement plus the margin; it holds its days for the fleet's owners and for every owner linked to it (below). An admission that settles with no uncertainty open on its fleet writes nothing permanent: the ordinary day of an unchanged fleet becomes complete. Discovery and its first hold are recorded in one transaction; if that transaction fails, a publication-visible *pending discovery* (the ledger's failure latch, extended with the lower bound) bars every day from that bound for every owner the fleet reaches until the write succeeds, and the fleet refuses every request meanwhile. Settlement is the last of the recorder's and the transport's completions, written by whichever lands last, with the upper bound no earlier than both. A fleet reaches, from its construction, every uncertainty and every unsettled admission of every ancestor not yet acknowledged, existing or discovered later; acknowledgement is root-first and carries the predecessor's whole effective set on; a fleet is never closed with a discovery of its own uncommitted. A successor acknowledges a predecessor only when the predecessor is terminal, and the acknowledgement links the successor's owners to every uncertainty of the predecessor still open, so a late extension reaches them. |
| **I3, one billing owner and one replication authority by file identity.** A file is billed to one `(account, app)` and replicated by one authority (the external replicator, the activator's hibernation path, or this fleet; today the first two exist and the fleet is unwired); any number of routes of that one owner may claim it through any spelling; two owners on one `(dev, ino)`, or two replication authorities, whether the file is listed or skipped, whether a claimant is live or hibernating, make it unowned (or refused for this fleet) with every claimant named, until an explicit handover of authority exists; every claimant's own authority check must pass; a claim with no identity yet is recorded and reconciled when it resolves; identity grouping is a look, not a snapshot, and relaxes none of today's hard-link, sidecar or filesystem checks; a database's lineage (which replica prefix continues) is a durable record in the shadow state, not an inode number; a required platform database that cannot be reached is an incomplete box. |
| @@ -18,7 +18,7 @@ The replicator (`internal/ops/litestream_metered.go`, `litestream_owners.go`; pa |
| **The primitive is a hand-walked, retained directory descriptor, not `os.Root`.** `os.Root` gives containment and re-resolves every name beneath its root; it follows links inside the root, `os.OpenRoot(path)` follows links in the path it is given, Go 1.26.5 walks per component with `openat` and there is no public constructor over a descriptor one has already verified. I1(b) and (c) need the descriptor itself. A small package, `internal/authority`: |
| -- `Dir`: a retained descriptor opened with `openat(parent, name, O_DIRECTORY|O_NOFOLLOW|O_RDONLY|O_CLOEXEC|O_NOCTTY)` (the kernel refuses a FIFO or device with `ENOTDIR` at lookup, before any open effect, which is the one existing directory-open test kept through the new shape), with its `(dev, ino, uid, mode)` read by `fstat` at acquisition and kept. Every method takes a basename (one component, not `.`, `..` or anything with a separator; anything else is refused before any syscall) and resolves it relative to this descriptor alone. `Child(name, expect) (*Dir, error)`: `openat` as above, then `fstat` compared with `expect` (the facts of the entry it is acquired from, when the caller has them); the child keeps a pointer to its parent `Dir` and its name, so the retained chain is a stack. `List() ([]Entry, error)`: a fresh open file description each time (`openat(fd, ".", O_DIRECTORY|O_RDONLY|O_CLOEXEC)`, `fstat`-compared with this descriptor, then `fdopendir` on it), never a `dup` (a `dup` shares the offset, so a second listing would start where the first ended); each entry's facts by `fstatat(fd, name, AT_SYMLINK_NOFOLLOW)`. `Readlink(name)`: `readlinkat`. `Walk(spelling, anchor)`: a captured link target, normalised first (repeated separators collapsed, `.` components dropped; a spelling that is empty after that is refused), then walked by components from an anchor: a relative spelling from this `Dir`, an absolute one from the pinned `/` root with the platform walk's rules (every component platform-owned) until it enters a pinned platform root, and from there as a relative walk beneath that root; a leading run of `..` pops the retained stack (never `openat(fd, "..")`, whose answer follows a moved directory to its new parent) and is refused when the stack would pass the anchor's root; a `..` after a named component is refused; each named component is `Child`; the last component is `fstatat(..., AT_SYMLINK_NOFOLLOW)` and its kind is returned: directory, regular, link, other (a FIFO, a socket, a device), absent, or error. The confinement boundary of a walk is the anchor's root: a relative walk cannot leave the tenant root it started in; an absolute walk must end beneath a pinned platform root or is refused. `OpenRegular(name, looked)`: `openat(fd, name, O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY|O_CLOEXEC)` then `fstat`, refused unless regular and equal to `looked`'s `(dev, ino)`. `Recheck()`: for this `Dir` and every ancestor on its stack, `fstatat(parent, name, AT_SYMLINK_NOFOLLOW)` compared with the retained child's facts; a difference anywhere is a changed edge. `Facts()`. |
| +- `Dir`: a retained descriptor opened with `openat(parent, name, O_DIRECTORY|O_NOFOLLOW|O_RDONLY|O_CLOEXEC|O_NOCTTY)` (the kernel refuses a FIFO or device with `ENOTDIR` at lookup, before any open effect, which is the one existing directory-open test kept through the new shape), with its `(dev, ino, uid, mode)` read by `fstat` at acquisition and kept. Every method takes a basename (one component, not `.`, `..` or anything with a separator; anything else is refused before any syscall) and resolves it relative to this descriptor alone. `Child(name, expect) (*Dir, error)`: `openat` as above, then `fstat` compared with `expect` (the facts of the entry it is acquired from, when the caller has them); the child keeps a pointer to its parent `Dir` and its name, so the retained chain is a stack. `List() ([]Entry, error)`: a fresh open file description each time (`openat(fd, ".", O_DIRECTORY|O_RDONLY|O_CLOEXEC)`, `fstat`-compared with this descriptor, then `fdopendir` on it), never a `dup` (a `dup` shares the offset, so a second listing would start where the first ended); each entry's facts by `fstatat(fd, name, AT_SYMLINK_NOFOLLOW)`. `Readlink(name)`: `readlinkat`. `Walk(spelling, anchor)`: a captured link target, normalised first (repeated separators collapsed, `.` components dropped, a trailing separator remembered as a requirement that the result be a directory; a spelling that was empty before normalisation is refused, and one that is only `.` or only a run of `..` resolves to the anchor or the popped directory as a directory result, which is what today's predicate does with `.`), then walked by components from an anchor: a relative spelling from this `Dir`, an absolute one from the pinned `/` root with the platform walk's rules (every component platform-owned) until it enters a pinned platform root, and from there as a relative walk beneath that root; a leading run of `..` pops the retained stack (never `openat(fd, "..")`, whose answer follows a moved directory to its new parent) and is refused when the stack would pass the anchor's root; a `..` after a named component is refused; each named component is `Child`; the last component is `fstatat(..., AT_SYMLINK_NOFOLLOW)` and its kind is returned: directory, regular, link, other (a FIFO, a socket, a device), absent, or error; a spelling with a trailing separator whose last component is not a directory is an error (`regular-file/` does not become the file). The confinement boundary of a walk is the anchor's root: a relative walk cannot leave the tenant root it started in; an absolute walk must end beneath a pinned platform root or is refused. `OpenRegular(name, looked)`: `openat(fd, name, O_RDONLY|O_NOFOLLOW|O_NONBLOCK|O_NOCTTY|O_CLOEXEC)` then `fstat`, refused unless regular and equal to `looked`'s `(dev, ino)`. `Recheck()`: for this `Dir` and every ancestor on its stack, `fstatat(parent, name, AT_SYMLINK_NOFOLLOW)` compared with the retained child's facts; a difference anywhere is a changed edge. `Facts()`. |
| - `Root`: a `Dir` reached by a walk from `/` (`open("/")` then `Child` per component), each component required to be the platform's (root's or this process's user's), not writable by group or others; the walk's `(dev, ino)` list is the root's *ancestry*. A root is built once per fleet and kept; `lookNow` rebuilds the ancestry from `/` and compares it, and a difference is a change the fleet goes stale on. |
| - Tenant roots: a unit route's tree is reached from a platform root through the platform-link predicate exactly as `resolvePlatformLinks` has it today (the link itself the platform's, its directory the platform's, its target's `..` only in a leading run, the target's parent the platform's, no further link in the target), evaluated on descriptors: the link read with `Readlink` from its directory's `Dir`, its target walked with `Walk` (anchored as the spelling says: relative from the link's directory, absolute from the pinned `/`), and the resulting `Dir` the tenant root; that predicate is one function, kept separate from the inventory's classification of links (below), and its restrictions do not change here. A file claim's chain top is found as today (upward by ownership from the file's directory) but every directory of the chain is then acquired downward from the platform root as `Dir`s, and the chain is the file's retained stack. Astra 2's layout (a tenant-owned component above a platform-owned one) fails the platform walk; no root is built over it. |
| @@ -34,23 +34,23 @@ Two layers. |
| 1. **The fork's own file operations** (`db.go`: the long-lived descriptor, WAL reads, shadow WAL, checkpoints, the `Stat`s of the database and its directory; `replica.go:186` and every LTX, temporary and cleanup path; enumerated in the plan's task by reading every call, not by sampling) take two retained directories, `SetDataDir(*authority.Dir, name)` for the database's directory and `SetMetaDir(*authority.Dir)` for the platform's shadow-state directory, and become `openat`/`fstatat`/`renameat`/`unlinkat` relative to them with `O_NOFOLLOW`; a sidecar is opened with `O_NONBLOCK` and refused unless `fstat` says regular (a FIFO put in a WAL's place is refused, not read). `db.f` is opened first through `OpenRegular` against the accepted facts and its identity is the database's for the life of the `DB`; before every reopen or sidecar open the fork compares `fstatat(dirfd, name)` with that identity and refuses on a difference (an error the fleet treats as a change). |
| -2. **SQLite's own opens, through a VFS this design owns whole, in a process of its own.** POSIX record locks belong to a process, and SQLite coordinates connections within one process through its own in-memory bookkeeping (`unixInodeInfo`: a second VFS in the same process reproducing the byte ranges would take a lock the ordinary connection believes it holds). So the fork and its VFS run in a **helper process** (`carlos-replicate`, one per fleet), which is the only SQLite user in its process: the agent builds the inventory, holds the roots and the witness, and admits requests; it hands the helper, over a unix socket with `SCM_RIGHTS`, the accepted immediate directory descriptor and name of each database, the shadow-state directory descriptor, and a socket on which the helper asks for each admission (the gate crosses the boundary as a request and an answer; the admission row is the agent's to write) and reports each transport's end; the helper's SQLite connections then coordinate with the application's and the agent's through the kernel's locks, as two processes do today, and the fleet's close is the helper's exit (which also ends every descriptor the helper held). A helper that exits unexpectedly is a fleet that died: the agent sees the socket close and treats it as a change (stale, hold, rebuild). The library opens the database, journal, WAL and SHM by pathname through its VFS after `xFullPathname` has canonicalised what it was given, and the default Unix VFS's *file* methods also act by pathname on their own: `xShmMap` opens the `-shm` file by deriving its name from the file's stored path and calling `open` directly, the SHM's removal calls the syscall table directly, the moved-file check stats the stored path, and the directory sync opens the directory by path. So delegating file methods to the Unix VFS is not confinement, and revision 2's delegation is withdrawn. The connection is given a **descriptor-rooted VFS** whose `sqlite3_file` object and every method are this design's: `xFullPathname` returns the name unchanged; `xOpen` is `openat(dirfd, name, flags|O_NOFOLLOW|O_CLOEXEC|O_NOCTTY)` for the main database and for each derived name (journal, WAL, SHM; `O_NONBLOCK` and a regular-file `fstat` for the sidecars), and for the main database compares the opened `(dev, ino)` with the accepted facts; `xAccess` and `xDelete` are `faccessat` and `unlinkat` on the descriptor; `xRead`, `xWrite`, `xTruncate`, `xSync`, `xFileSize` act on the descriptor; `xLock`, `xUnlock` and `xCheckReservedLock` are the POSIX advisory locks the Unix VFS uses, on the descriptor, with the same byte ranges, so the application's own connection and this one agree; `xShmMap`, `xShmLock`, `xShmBarrier` and `xShmUnmap` open the `-shm` through `openat` on the directory descriptor and `mmap` it, with the Unix VFS's lock byte layout, so the WAL index is shared correctly with the application; `xSync` of the directory is `fsync` of the retained directory descriptor; `xFileControl` answers the controls the library asks of a WAL-mode file (`SQLITE_FCNTL_PERSIST_WAL`, `SQLITE_FCNTL_POWERSAFE_OVERWRITE`, `SQLITE_FCNTL_SIZE_HINT`, `SQLITE_FCNTL_HAS_MOVED` answered from `fstatat(dirfd, name)` against the retained identity) and `SQLITE_NOTFOUND` for the rest; no method takes a pathname from the stored name after `xOpen`. `xAccess` answers from `fstatat(dirfd, name, AT_SYMLINK_NOFOLLOW)` and reports a link as absent rather than escaping through it; `xOpen` of a temporary or null-name file (the library's transient files) is answered in the shadow-state directory descriptor, never the tenant's; permitted names are the main name and its four derived names, and any other name is refused; `xClose` closes the descriptor and releases the locks the object held; `xDeviceCharacteristics` reports the same as the Unix VFS for the filesystem; URI parameters are escaped as SQLite's URI grammar requires. The DSN is `file:<name>?vfs=carlos-dirfd-<id>` and the driver passes `vfs` through: that selects the VFS and confines nothing by itself; the confinement is the validated names and the descriptor-relative operations. |
| +2. **SQLite's own opens, through a VFS this design owns whole, in a process of its own.** POSIX record locks belong to a process, and SQLite coordinates connections within one process through its own in-memory bookkeeping (`unixInodeInfo`: a second VFS in the same process reproducing the byte ranges would take a lock the ordinary connection believes it holds). So the fork and its VFS run in a **helper process** (`carlos-replicate`, one per fleet), in which every SQLite connection is one of the VFS's own, and the VFS keeps the within-process bookkeeping SQLite's Unix VFS keeps (litestream holds a read transaction on one pooled connection while working through another, so the helper has several): one shared record per inode in the helper, with the count of connections holding each shared lock, local exclusion checked before an exclusive lock is taken (as `unixInodeInfo` does), a POSIX unlock issued only when the last local holder releases, and a descriptor on an inode closed only when no object of that inode holds a lock (deferred otherwise), with the fork's raw descriptors (`db.f`, the WAL reads) opened through the same bookkeeping so their closes are deferred the same way. In that arrangement: the agent builds the inventory, holds the roots and the witness, and admits requests; it hands the helper, over a unix socket with `SCM_RIGHTS`, the accepted immediate directory descriptor and name of each database, the shadow-state directory descriptor, and a socket on which the helper asks for each admission (the gate crosses the boundary as a request and an answer; the admission row is the agent's to write) and reports each transport's end; the helper's SQLite connections then coordinate with the application's and the agent's through the kernel's locks, as two processes do today, and the fleet's close is the helper's exit (which also ends every descriptor the helper held). A helper that exits unexpectedly is a fleet that died: the agent sees the socket close, then waits for the child's exit (`waitpid`, with `SIGKILL` after a bound), and only then, with the helper proven gone, settles each of the helper's admissions that is not terminal (whatever participant state it is in: nothing done, transport done, or recorder done) as `recovered-unknown` with the missing timestamps `now`, opening and closing an uncertainty for each as recovery does, in idempotent transactions it retries until they commit, and then rebuilds; `recovered-unknown` is reachable from every nonterminal state, not from `admitted` alone. The library opens the database, journal, WAL and SHM by pathname through its VFS after `xFullPathname` has canonicalised what it was given, and the default Unix VFS's *file* methods also act by pathname on their own: `xShmMap` opens the `-shm` file by deriving its name from the file's stored path and calling `open` directly, the SHM's removal calls the syscall table directly, the moved-file check stats the stored path, and the directory sync opens the directory by path. So delegating file methods to the Unix VFS is not confinement, and revision 2's delegation is withdrawn. The connection is given a **descriptor-rooted VFS** whose `sqlite3_file` object and every method are this design's: `xFullPathname` returns the name unchanged; `xOpen` is `openat(dirfd, name, flags|O_NOFOLLOW|O_CLOEXEC|O_NOCTTY)` for the main database and for each derived name (journal, WAL, SHM; `O_NONBLOCK` and a regular-file `fstat` for the sidecars), and for the main database compares the opened `(dev, ino)` with the accepted facts; `xAccess` and `xDelete` are `faccessat` and `unlinkat` on the descriptor; `xRead`, `xWrite`, `xTruncate`, `xSync`, `xFileSize` act on the descriptor; `xLock`, `xUnlock` and `xCheckReservedLock` are the POSIX advisory locks the Unix VFS uses, on the descriptor, with the same byte ranges, so the application's own connection and this one agree; `xShmMap`, `xShmLock`, `xShmBarrier` and `xShmUnmap` open the `-shm` through `openat` on the directory descriptor and `mmap` it, with the Unix VFS's lock byte layout, so the WAL index is shared correctly with the application; `xSync` of the directory is `fsync` of the retained directory descriptor; `xFileControl` answers the controls the library asks of a WAL-mode file (`SQLITE_FCNTL_PERSIST_WAL`, `SQLITE_FCNTL_POWERSAFE_OVERWRITE`, `SQLITE_FCNTL_SIZE_HINT`, `SQLITE_FCNTL_HAS_MOVED` answered from `fstatat(dirfd, name)` against the retained identity) and `SQLITE_NOTFOUND` for the rest; no method takes a pathname from the stored name after `xOpen`. `xAccess` answers from `fstatat(dirfd, name, AT_SYMLINK_NOFOLLOW)` and reports a link as absent rather than escaping through it; `xOpen` of a temporary or null-name file (the library's transient files) is answered in the shadow-state directory descriptor, never the tenant's; permitted names are the main name and its four derived names, and any other name is refused; `xClose` closes the descriptor and releases the locks the object held; `xDeviceCharacteristics` reports the same as the Unix VFS for the filesystem; URI parameters are escaped as SQLite's URI grammar requires. The DSN is `file:<name>?vfs=carlos-dirfd-<id>` and the driver passes `vfs` through: that selects the VFS and confines nothing by itself; the confinement is the validated names and the descriptor-relative operations. |
| What this closes: every open the library makes for the database and its sidecars, including reopens, checkpoints, the SHM and the first open's initialisation writes, happens beneath the accepted immediate directory with no link followed and no pathname above it walked, and the locks are coordinated with every other connection in the system by the kernel because no two SQLite instances share the helper's process. What it leaves, stated: within its own directory a tenant can still replace `<name>` or a sidecar with another regular file it owns between two opens; the identity comparison at `xOpen` and before each of the fork's opens refuses the main database's replacement; a sidecar swapped under a live connection is a corruption of the replica, which litestream's own consistency checks are the place to catch and which the plan's task for the fork adds a test for. That residual is "the owner's own doing" only under decision 12 (a foreign user's directory inside a tenant's tree can supply sidecars today); until 12 is decided it is stated as a residual with two possible authors. |
| -The VFS is the plan's first spike and it must prove, before anything else of Part B is written: registration from Go against this driver's generated API on amd64 and arm64; a database opened, written, checkpointed, closed and reopened through the VFS in one process while an ordinary connection in another process (the application's arrangement) and, as the negative case, an ordinary connection in the same process, write, read and checkpoint concurrently, with the locks observed to exclude correctly across processes and the same-process arrangement shown unsafe (which is why the helper exists); SHM creation and removal through the VFS; the moved-file check; closure releasing locks without disturbing the other process's; and a trace of every system call the library made (`strace` on the spike, not `fstat` alone, since identity does not show ancestry) with the list of pathname entry points that remain (expected: none for a connection opened with `SQLITE_OPEN_URI` and `vfs=`). The spike's result is a note, and if the spike fails on any of these the design's Part B is rewritten before the plan continues; Parts A, C and D do not wait on it. |
| +The VFS is the plan's first spike and it must prove, before anything else of Part B is written: registration from Go against this driver's generated API on amd64 and arm64; a database opened, written, checkpointed, closed and reopened through the VFS in one process while an ordinary connection in another process (the application's arrangement) and, as the negative case, an ordinary connection in the same process, write, read and checkpoint concurrently, with the locks observed to exclude correctly across processes and the same-process arrangement shown unsafe (which is why the helper exists); two of the VFS's own connections in the helper holding and releasing the same shared WAL lock while the outside connection tries an exclusive one, showing the count holds it until the last local release and a raw descriptor's close deferred while a lock stands; SHM creation and removal through the VFS; the moved-file check; closure releasing locks without disturbing the other process's; and a trace of every system call the library made (`strace` on the spike, not `fstat` alone, since identity does not show ancestry) with the list of pathname entry points that remain (expected: none for a connection opened with `SQLITE_OPEN_URI` and `vfs=`). The spike's result is a note, and if the spike fails on any of these the design's Part B is rewritten before the plan continues; Parts A, C and D do not wait on it. |
| ## Part C. The durable admission, two kinds of hold, and a handover on disk (Astra 3, 4) |
| **The admission is a row, written before the send.** The gate (`lease.RequestGate`) returns an *admission*, `(fleet_id, owner, token, admitted_at)`, after committing a row in the durability ledger's own database (`admissions`: `token`, `fleet_id`, `owner account and app`, `admitted_at`, `transport_done_at`, `settled_at`, `how`), `synchronous=FULL`, within a bounded budget; the gate waits for the commit. A commit that does not return within the budget refuses the request (`ErrAdmissionNotDurable`; litestream sees a failed upload and does not advance) and the goroutine that awaited it stays to see the commit land and then marks the row `how = unissued, settled_at = now` in its own transaction, so a late commit never authorises anything and never blocks a day; if the process dies between, recovery classifies the row as below. The meter's `Start` row carries the token (`Start` may still fail or be late, as today; what is now mandatory is the admission, not the `Start`). An admission has two participants, the transport and the recorder, and is settled by whichever finishes last. The transport writes `transport_done_at` from the SDK middleware's finalize step. The recorder writes `recorder_done_at` with its outcome: `finished` (`Finish`), `abandoned` (`Abandon`), or `unrecorded` (the request path, when `Start` failed and made no attempt row, which today calls neither; it now writes this terminal). The participant that finds the other already done writes `settled_at = max(transport_done_at, recorder_done_at)` and `how`, and performs, in that same transaction, the uncertainty extension or closure below. The terminals that have no participants (`unissued`, `recovered-unknown`) write all three timestamps at once. The transition table is the plan's model: `admitted` → (`transport done` | `recorder done`) → `settled`, with `unissued` and `recovered-unknown` as terminals from `admitted`, and no other edge. Admissions are the ledger's pending state moved to before the send and given an owner. |
| -**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `from = min(the registry proof, the last clean look's start, the earliest admitted_at of the fleet's admissions then unsettled) - margin`. If discovery finds no admission of the fleet unsettled, the uncertainty is opened and closed in that transaction (`to = discovery + margin`). While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for every day of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it with `to = max(discovery, that settlement) + margin` and materialises every day of the closed interval, the margin's extra day across midnight included; recovery uses the same helper and advances the publication watermark past the days it materialised. An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval, whichever participant finished last; the request completed before the change was noticed is inside the lower bound as today. **If the discovery transaction fails** the barrier is not enough (every admission may already be settled): discovery first latches a *pending discovery* in the ledger's existing failure latch (`latchFailure`, extended to carry the lower bound and the owners reached), which the publisher consults (`drainFailures` before any snapshot) and which bars every day from the bound for those owners until the uncertainty commits; the fleet refuses every request meanwhile and retries the write; a process that dies with the latch held leaves no durable row, and recovery's uncertainty for a fleet not `closed` (below) covers it from the fleet's `opened_at`. |
| +**Two kinds of hold.** A *barrier* is the ledger's completeness rule: a day is not published complete for an admission's owner while an admission of that owner admitted on or before that day is unsettled (box-wide while Part C2's owner scope is not in use); it is a query over `admissions`, writes nothing, and lifts at settlement. An *uncertainty* is a row (`uncertainties`: `id`, `fleet_id`, `from`, `to` null while open, `reason`) written when a fleet's authority is found changed (by the witness, by `Recheck`, by a changed file), in the transaction that also writes the holds for the days of `[from, now]` for every owner the uncertainty reaches: `from = min(the registry proof, the last clean look's start, the earliest admitted_at of the fleet's admissions then unsettled) - margin`. If discovery finds no admission of the fleet unsettled, the uncertainty is opened and closed in that transaction (`to = discovery + margin`). While an uncertainty is open, settlements of that fleet's admissions extend it: each settlement, in its own transaction, writes the holds for every day of `[its admitted_at, its settled_at]` for every owner the uncertainty reaches, and the last settlement closes it with `to = max(discovery, that settlement) + margin` and materialises every day of the closed interval, the margin's extra day across midnight included; recovery uses the same helper and advances the publication watermark past the days it materialised. An admission that settles while no uncertainty of its fleet is open writes nothing permanent: the ordinary day of an unchanged fleet becomes complete when its barrier lifts. The multiple-midnights sequence (admitted on the 2nd, the change at 23:58:10, sent on the 3rd, finished on the 4th) holds the 2nd, 3rd and 4th through the settlement's interval, whichever participant finished last; the request completed before the change was noticed is inside the lower bound as today. **If the discovery transaction fails** the barrier is not enough (every admission may already be settled): discovery first latches a *pending discovery* in the ledger's in-memory failure overlay (`latchFailure`, extended to carry the lower bound and the owners reached), which the publisher's snapshot already overlays and detects across a drain (today's `snapshotOnce` and `drainFailures` behaviour, extended rather than replaced: a failed drain must not stop the overlay from making the snapshot incomplete), so every day from the bound reads incomplete for those owners until the uncertainty commits; the fleet refuses every request meanwhile and retries the write; the fleet cannot be marked `closed` while the discovery is uncommitted (above); a process that dies with the latch held leaves a fleet not `closed`, which recovery covers from the fleet's `opened_at`. |
| -**Who an uncertainty reaches, and what a fleet inherits.** A fleet's *effective* uncertainties are its own and every one it inherited. Links are made at **construction**, not at acknowledgement: a successor, in the transaction that writes its own `open` row, links itself (`uncertainty_links`: `uncertainty_id`, `fleet_id`) to every open uncertainty of its predecessor's effective set, and to its predecessor's unsettled admissions as a barrier source (`barrier_links`: `predecessor fleet_id`, `fleet_id`), so from its first instant a late extension of the predecessor's uncertainty writes holds for the successor's owners and the predecessor's unsettled admissions bar the successor's days. Acknowledgement is the later, terminal step (below) and carries the predecessor's whole effective set, closed intervals included, into the successor's inherited set (`inherited`: `fleet_id`, `uncertainty_id`), so the next successor inherits it in turn: A's U reaches B at B's acknowledgement of A and C at C's acknowledgement of B, as `staleWindow` passes an inherited window on today. Astra's October sequence under owner scope: Y, built at 23:58:20, links at construction to A's open uncertainty and to A's unsettled admission; on the 3rd Y's day is barred through the barrier link; on the 4th the settlement writes the 2nd to 4th for X and, through the link, for Y. |
| +**Who an uncertainty reaches, and what a fleet inherits.** Reach is computed over the chain, not stored per uncertainty: every fleet row names its predecessor, and a fleet's *unacknowledged ancestry* is the set of ancestors reached by following predecessors through fleets that have not been acknowledged (an acknowledged step ends the walk, because acknowledgement copies). An uncertainty, whenever it is opened, extended or closed, writes holds for the owners of its fleet and of every fleet whose unacknowledged ancestry contains that fleet (a query at write time, so an uncertainty discovered after a successor's construction reaches it, and so does one on a grandparent when the parent closed unopened without acknowledging); the barrier for an owner consults the unsettled admissions of its fleet and of its whole unacknowledged ancestry. Acknowledgement is terminal and **root-first**: a fleet may acknowledge its predecessor only when the predecessor is `closed` and has itself acknowledged its own predecessor (or has none), and it copies the predecessor's effective set (own and inherited, open and closed) into its inherited set (`inherited`: `fleet_id`, `uncertainty_id`) and writes holds for the closed ones its owners have not been held for, in one transaction; after that the chain walk stops there and the copy carries on. Astra's sequence: A has U open and an admission unsettled; B is built (its ancestry: A) and closes unopened without acknowledging; C is built against B (its ancestry: B, then A, since B has not acknowledged); on the 4th A's settlement extends U and the write reaches X, B's owners and C's owners through the ancestry; when A closes, B acknowledges A (root-first), then C acknowledges B and inherits U. Under owner scope the October sequence: Y, built at 23:58:20 with A in its ancestry, is barred on the 3rd by A's unsettled admission and held on the 4th by the settlement's write. |
| -**The fleet journal and the handover.** One row per fleet (`fleets`: `fleet_id`, `box`, `witness_generation`, `owners`, `opened_at`, `state`, `predecessor`, `acknowledged_by`). States and the transaction each is written in: `open` at build, before any admission, with the construction links above; `stale` when an uncertainty is opened; `closing` when the helper has exited and the fleet has released its witness; `closed` when every admission of the fleet is settled (the recorder exposes a wait on settlement; `closeNow` waits, bounded, and past the bound the row stays `closing`, with its admissions still barring and its uncertainties still open: a late worker is a late settlement, and nothing about it is lost). A successor acknowledges a predecessor only when the predecessor is `closed`, or when recovery has closed it; acknowledgement writes `acknowledged_by`, copies the predecessor's effective uncertainties (its own and inherited, open and closed) into the successor's inherited set, and writes holds for the closed ones the successor's owners have not yet been held for, in one transaction. A `closed` predecessor whose effective set is empty contributes nothing: routine replacement of a healthy chain holds no day. An unopened intermediate fleet (built and closed without replicating) has its construction links and is acknowledged like any other, so the chain's barriers and inheritance pass through it. |
| +**The fleet journal and the handover.** One row per fleet (`fleets`: `fleet_id`, `box`, `witness_generation`, `owners`, `opened_at`, `state`, `predecessor`, `acknowledged_by`). States and the transaction each is written in: `open` at build, before any admission, with the construction links above; `stale` when an uncertainty is opened; `closing` when the helper has exited and the fleet has released its witness; `closed` when every admission of the fleet is settled and no discovery of the fleet is uncommitted (a discovery whose write failed is retried, and the closure transaction carries any still-pending discovery's uncertainty with it, so a fleet is closed with that uncertainty committed or not closed at all; the recorder exposes a wait on settlement; `closeNow` waits, bounded, and past the bound the row stays `closing`, with its admissions still barring and its uncertainties still open: a late worker is a late settlement, and nothing about it is lost). Before the helper is told to stop, `closeNow` keeps today's ordering: replication stops first, then one last look at the files and one last reading of the witness, then the witness is released; a change found by that last look is a discovery like any other. A successor acknowledges a predecessor only when the predecessor is `closed`, or when recovery has closed it; acknowledgement writes `acknowledged_by`, copies the predecessor's effective uncertainties (its own and inherited, open and closed) into the successor's inherited set, and writes holds for the closed ones the successor's owners have not yet been held for, in one transaction. A `closed` predecessor whose effective set is empty contributes nothing: routine replacement of a healthy chain holds no day. An unopened intermediate fleet (built and closed without replicating) has its construction links and is acknowledged like any other, so the chain's barriers and inheritance pass through it. |
| -**The producer fence, and recovery at process start.** The ledger's own lock does not prove the old process gone: `DurabilityLedger.Close` releases it when its bounded wait ends with work in flight, and a request admitted before could still be sent after. So dispatch is fenced separately: a *producer fence*, an `flock` on `admissions.lock` in the ledger's directory, taken by the agent process before its first admission and held for the process's life (never released by a timeout; released only by the kernel at exit), and every admission's commit requires it held (checked under the ledger's mutex before `BEGIN`); the helper process cannot dispatch without an admission, and the admission cannot be written without the fence, so a process that has lost the fence cannot cause a send. Recovery takes the fence first and, holding it, in one transaction before any fleet is built: every admission of this box that is unsettled is classified `how = recovered-unknown` with all three timestamps `now`, an uncertainty opened and closed for it at once over `[admitted_at - margin, now + margin]` for its owner (box-wide while owner scope is not in use), its days materialised; every fleet not `closed` is marked `closed` with an uncertainty over `[min(opened_at, its earliest admission) - margin, now + margin]` if it had none open, or its open uncertainty closed at `now + margin` and materialised; every fleet with no `acknowledged_by` is acknowledged by the first fleet the new process builds, with the inheritance and holds above. Orphans therefore terminate, conservatively, and nothing blocks beyond its recovery; a producer that is somehow still alive cannot send, because the fence is no longer its. |
| +**The producer fence, and recovery at process start.** The ledger's own lock does not prove the old process gone: `DurabilityLedger.Close` releases it when its bounded wait ends with work in flight, and a request admitted before could still be sent after. So dispatch is fenced separately: a *producer fence*, an `flock` on `admissions.lock` in the ledger's directory, taken by the agent before its first admission on an open file description it then passes to every helper it starts (over the same `SCM_RIGHTS` message as the directories, before the helper may send anything; a helper that has not received it refuses to dispatch), never released by a timeout and released by the kernel only when the agent and every helper holding that description have exited; every admission's commit requires the fence held (checked under the ledger's mutex before `BEGIN`). A helper that outlives its agent with an admission in hand therefore still holds the fence, and a new agent cannot take it, and so cannot recover, until that helper is gone; a helper's exit is observed by the agent as its child's `waitpid`, never inferred from a closed socket, and `PR_SET_PDEATHSIG` is set in the helper as a convenience that shortens the wait and proves nothing. Recovery requires the fence, which is the verified termination of every old producer. Recovery takes the fence first and, holding it, in one transaction before any fleet is built: every admission of this box that is unsettled is classified `how = recovered-unknown` with all three timestamps `now`, an uncertainty opened and closed for it at once over `[admitted_at - margin, now + margin]` for its owner (box-wide while owner scope is not in use), its days materialised; every fleet not `closed` is marked `closed` with an uncertainty over `[min(opened_at, its earliest admission) - margin, now + margin]` if it had none open, or its open uncertainty closed at `now + margin` and materialised; every fleet with no `acknowledged_by` is acknowledged, root-first, by the first fleet the new process builds, with the inheritance and holds above. Orphans therefore terminate, conservatively, and nothing blocks beyond its recovery; a producer that is somehow still alive cannot send, because the fence is no longer its. |
| **Part C2, who is held.** The ledger's failure rows and `Incomplete` are box-wide today (`DurabilityClient.Incomplete` discards the client's identity; `CompletenessFor(account, …)` fails on any box failure). Owner-scoped holds are a ledger change: a `holds` table `(day, account, app or empty for the box, uncertainty_id, written_at)`, the publisher's snapshot carrying them, the reader treating a box-wide row as failing every owner and an owner row as failing that owner alone, and older readers of a published day seeing a box-wide incompleteness marker whenever any hold exists (so an older console never reads a held day as complete). The design specifies the table, the reader and the marker and keeps **box-wide the default**: every hold this design writes names the box until decision 10 is taken. Decision 10's implementation boundary is also stated, so it is not mistaken for a switch: per-tenant refusal means per-owner fleets (a fleet per `(account, app)`, each with its own witness view, uncertainties and holds, and a tenant's unacceptable file refusing only its fleet), which is a change to `MeteredFleetStore`'s shape beyond the holds table. |
| @@ -60,11 +60,11 @@ The VFS is the plan's first spike and it must prove, before anything else of Par |
| **Replication authority.** Separately from the owner, each identity has at most one replication authority: the external replicator (the box today), the activator's hibernation path (which replicates a hibernating exec route's database and holds its own stream), or this fleet. The fleet replicates an identity only when no other authority has it: a hibernating exec route whose database resolves to the same identity as a live unit tree's listed file is a conflict of authorities, refused for this fleet with both named, exactly as today's live/hibernating conflict is, until an explicit handover (a route's hibernation path releasing the database to the fleet) exists, which is not designed here. Deduplicating two spellings of one identity within the fleet is allowed only when both claimants are routes this fleet would replicate. |
| -**Lineage.** A replica key today is `(account, path)`, and an inode number is an observation, not a database: a deleted and recreated file can reuse it across a restart. The lineage is therefore a durable record in the shadow state (platform-owned): per replica key, `lineage_id` (random), `owner`, the accepted spelling, `(dev, ino)`, the file's birth time (`statx` `btime`, where the filesystem has it) and the SQLite header's database id fields read at acceptance. Within one fleet's life, another accepted spelling of the same `(dev, ino)` reuses the key (one stream, one file). Across a restart, a key is resumed only when `(dev, ino)` and birth time match the record (and the header's id fields do not contradict it); otherwise the file is a new database and a new lineage (a new prefix) after the hold the change causes. Existing path-based prefixes are adopted as the first lineage of the file found under that path at the first build after this design, recorded then with its identity and birth time; a later mismatch rotates as above. |
| +**Lineage.** A replica key today is `(account, path)`, and an inode number is an observation, not a database: a deleted and recreated file can reuse it across a restart. The lineage is therefore a durable record in the shadow state (platform-owned; one record per replica key, written with the shadow state's own fsync-and-rename discipline before the database is opened for replication, so no replica write precedes its lineage): `lineage_id` (random), `owner`, the accepted spelling, `(dev, ino)` and the file's birth time as `statx` reports it with `STATX_BTIME` set. Continuity means the same filesystem incarnation of the file, nothing more: within one fleet's life another accepted spelling of the same `(dev, ino)` reuses the key (one stream, one file); across a restart a key is resumed only when `(dev, ino)` and a valid birth time match the record; a filesystem that reports no valid birth time, or a mismatch, rotates to a new lineage (a new prefix) after the hold the change causes. A database replaced in place over the same incarnation (a restore written into the same inode) is not distinguished by this record and is not claimed to be; it is the owner's own replacement and litestream's generation handling continues the stream. No SQLite header field identifies a database incarnation and none is used. Existing path-based prefixes are adopted as the first lineage of the file found under that path at the first build after this design, recorded then; a later mismatch rotates as above. |
| **Required coverage.** The platform's required databases are a list: the registry this process has open, the console database when the agent's role configuration says the box is a console box, `CONSOLE_DB` when set. A required database unreachable through its root is a problem that makes the inventory incomplete and is written as a durable box-wide hold for the day before and independent of whether a fleet is built. Before any cutover, a read-only operator command lists this inventory beside the external replicator's configuration. |
| -**Route identity (Astra 10, approval H): a producer and a lifecycle, not a number.** The current whole-app backend starts a unit and checks its health; it reads no uid or invocation, units restart on their own (`Restart=always`, socket activation), and a stale pair would survive an autonomous restart. The design names the mechanism: the identity of a route's running workload is read from systemd, not inferred: the unit observer the activator already runs for isolated exec is extended to every unit route and reads, at every start, stop and restart it is told of (D-Bus `PropertiesChanged` on the unit, with a poll of `systemctl show -p InvocationID,MainPID,ActiveState` as the fallback), the `InvocationID` and the main process's uid from `/proc/<MainPID>/status`; it writes `route_uid` and `invocation_id` on the route row, under the ownership witness (the generation covers the columns), as a conditional write keyed on the previously recorded invocation (a write that finds another invocation recorded clears both and lets the next observation set them); it clears both at stop. A claim is honoured only when the route's recorded invocation is current (the observer's last reading agrees with systemd now) and the claim root's owning uid equals `route_uid`; unknown identity refuses the claim. Hibernation capability is a route property; a hibernating route that is awake has an invocation and claims as a live unit; one that is asleep has none, and reaching its private directory through the platform root proves which directory was reached, not that this route owns it, so a sleeping route claims only against a **binding** the observer recorded while it was awake: `state_dir_identity`, the `(dev, ino)` of the state directory the live workload's uid owned at the observer's last coherent reading, written on the route row under the witness beside `route_uid` and `invocation_id` and kept across sleep; a sleeping route with no binding, or whose private spelling resolves to another identity, is refused. A coherent reading is: `InvocationID` and `MainPID` from systemd, the uid from `/proc/<MainPID>/status`, then `InvocationID` and `MainPID` from systemd again, equal to the first; a reading that is not coherent is retried and, failing, leaves the route unknown. Freshness is checked where claims are made: at the inventory's build and at every look (`lookNow`), by comparing the recorded invocation with systemd's current one; a route whose invocation changed between is a change the fleet goes stale on, since the registry row changed under the witness. An exec or named-file claim of a non-unit route (the activator's own processes) is claimed under the activator's identity, which is a trusted workload and so a parameter. Which uids are the platform's own workloads, and whether a root-run unit may claim, are `trustedUIDs` (empty) and approval H. |
| +**Route identity (Astra 10, approval H): a producer and a lifecycle, not a number.** The current whole-app backend starts a unit and checks its health; it reads no uid or invocation, units restart on their own (`Restart=always`, socket activation), and a stale pair would survive an autonomous restart. The design names the mechanism: the identity of a route's running workload is read from systemd, not inferred: the unit observer the activator already runs for isolated exec is extended to every unit route and reads, at every start, stop and restart it is told of (D-Bus `PropertiesChanged` on the unit, with a poll of `systemctl show -p InvocationID,MainPID,ActiveState` as the fallback), the `InvocationID` and the main process's uid from `/proc/<MainPID>/status`; it writes `route_uid` and `invocation_id` on the route row, under the ownership witness (the generation covers the columns), as a conditional write keyed on the previously recorded invocation (a write that finds another invocation recorded clears both and lets the next observation set them); it clears both at stop. A claim is honoured only when the route's recorded invocation is current (the observer's last reading agrees with systemd now) and the claim root's owning uid equals `route_uid`; unknown identity refuses the claim. Hibernation capability is a route property; a hibernating route that is awake has an invocation and claims as a live unit; one that is asleep has none, and reaching its private directory through the platform root proves which directory was reached, not that this route owns it, so a sleeping route claims only against a **binding** the observer recorded while it was awake: `state_dir_identity`, the `(dev, ino, birth time)` of the state directory the live workload's uid owned at the observer's last coherent reading, with that uid and the platform link's target spelling, written on the route row under the witness beside `route_uid` and `invocation_id` and kept across sleep; the binding is invalidated by any change to the route the witness sees and by a state change the observer sees (a stop that removes the directory); a sleeping route with no binding, whose private spelling resolves to a directory of another identity or birth time or owner, or whose filesystem reports no valid birth time, is refused: continuity that cannot be established is refused, as with lineage below. A coherent reading is: `InvocationID` and `MainPID` from systemd, the uid from `/proc/<MainPID>/status`, then `InvocationID` and `MainPID` from systemd again, equal to the first; a reading that is not coherent is retried and, failing, leaves the route unknown. Freshness is checked where claims are made: at the inventory's build and at every look (`lookNow`), by comparing the recorded invocation with systemd's current one, and a mismatch stales the fleet directly (the observer may not have written the change yet, so the witness is not relied on for it); it is not checked per admission, which is stated as the bound: a unit restarted between two looks is caught at the next look, within the watch interval, and the margin covers the look. An exec or named-file claim of a non-unit route (the activator's own processes) is claimed under the activator's identity, which is a trusted workload and so a parameter. Which uids are the platform's own workloads, and whether a root-run unit may claim, are `trustedUIDs` (empty) and approval H. |
| ## The parameters this design keeps open |
diff --git a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md| index 7de127e98..fbb649405 100644 |
| --- a/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md |
| +++ b/docs/superpowers/specs/2026-10-03-replicator-pinned-roots-spikes.md |
| @@ -1,15 +1,15 @@ |
| # Spike notes for the replicator's authority design |
| -Read-only findings (no code written or run), 2026-10-03 06:45, for plan step 0 of `2026-10-03-replicator-pinned-roots-design.md`. |
| +Read-only findings (no code written or run), 2026-10-03 06:45, revised 07:25 for revision 5 of `2026-10-03-replicator-pinned-roots-design.md`. Two conclusions of the first version are superseded and struck below: that the Unix VFS's file methods could be delegated (they open the SHM and check moved files by pathname; the VFS owns every method now) and that `fstat` shows ancestry (it shows identity; the spike traces system calls). |
| ## The descriptor-rooted VFS on `modernc.org/sqlite` v1.55.0 |
| - The driver ships a Go-registered VFS: `modernc.org/sqlite/vfs` registers a VFS table with `sqlite3.Xsqlite3_vfs_register` from Go (`vfs/patches.go:195`) and keeps Go state in `pAppData`. Its method table is a C source (`vfs/c/vfs.c`, 713 lines: `vfsOpen`, `vfsDelete`, `vfsAccess`, `vfsFullPathname`, `vfsRead`, `vfsWrite`, and locking that is a no-op) compiled through the same toolchain as the library, over an `fs.FS` and read-oriented. |
| -- So the route exists: a VFS implemented outside the library, registered by name, chosen per connection with `?vfs=<name>`. What ours must differ in: `xOpen` as `openat` from a retained directory descriptor with `O_NOFOLLOW|O_CLOEXEC|O_NOCTTY` for the main database and each derived name (journal, WAL, SHM); `xAccess` and `xDelete` as `faccessat` and `unlinkat` on it; `xFullPathname` returning the name unchanged (so no canonicalisation walks a tenant's ancestry); real `xLock`/`xUnlock`/`xCheckReservedLock` and `xShmMap`/`xShmLock`/`xShmBarrier`/`xShmUnmap` (WAL mode needs them; the demo's no-op locks are not acceptable), which can delegate to the default Unix VFS's file methods once the descriptor is open, since those operate on the file descriptor, not the path. |
| +- So the route exists: a VFS implemented outside the library, registered by name, chosen per connection with `?vfs=<name>`. What ours must differ in: `xOpen` as `openat` from a retained directory descriptor with `O_NOFOLLOW|O_CLOEXEC|O_NOCTTY` for the main database and each derived name (journal, WAL, SHM); `xAccess` and `xDelete` as `faccessat` and `unlinkat` on it; `xFullPathname` returning the name unchanged (so no canonicalisation walks a tenant's ancestry); real `xLock`/`xUnlock`/`xCheckReservedLock` and `xShmMap`/`xShmLock`/`xShmBarrier`/`xShmUnmap` (WAL mode needs them; the demo's no-op locks are not acceptable), written by us with per-inode bookkeeping across the helper's own connections (shared-lock counts, local exclusion, deferred descriptor close), since the Unix VFS's versions open the SHM and check moved files by pathname and cannot be delegated to. |
| - Revision 3's review added what the spike must settle about locking: SQLite coordinates connections within one process through its own per-inode bookkeeping (`unixInodeInfo`), so a second VFS in the same process cannot exclude an ordinary connection by reproducing POSIX byte ranges; the design therefore runs the fork and its VFS in a helper process per fleet. The spike's deliverable, when run: a throw-away helper program that registers such a VFS, receives a directory descriptor over a unix socket (`SCM_RIGHTS`), opens a database and its WAL through it, writes, checkpoints, closes and reopens, while a second program holds an ordinary connection and writes, reads and checkpoints concurrently, with the locks observed to exclude correctly across the two processes; the same-process arrangement run as the negative case and shown unsafe; SHM created and removed through the VFS; the moved-file check; closure releasing only the helper's locks; a system-call trace (`strace -f`) of the library's operations rather than `fstat` alone, since identity does not show ancestry; and the list of library entry points that take a pathname outside the VFS (none are expected for the connection; `sqlite3_open_v2` with `SQLITE_OPEN_URI` and the `vfs` parameter is the only path in). On amd64 and arm64. |
| - Risk noted: the generated library's VFS structs are per-platform (`vfs_linux_amd64.go` and siblings); a VFS written in Go against them is per-platform too, and the agent builds for linux/arm64 as well as amd64 (the cross builds in the gate). |
| ## The admission's cost on the request path |
| - `DurabilityClient.Start` (`internal/usage/durabilityledger.go:1007`) is already one transaction per attempt on a `synchronous=FULL` WAL database: an `INSERT` into `attempts`, `touchDay`, `noteInstant`, `persistSeq`, commit. The meter gives it `requestMeterBudget` (25 ms) and sends regardless of its outcome (`internal/lease/requestmeter.go`). |
| -- The design's admission is that same commit moved to before permission: the gate's answer waits for it, and a commit that misses the budget refuses the request instead of sending anyway (the late commit is then marked unissued by the goroutine that awaited it). The cost is therefore not a second commit but a dependency: the request path's latency includes the ledger's commit, and a slow disk refuses requests rather than sending unrecorded ones. With the helper process the admission crosses a socket as well. The spike's deliverable: the ledger's commit latency distribution on a box's disk under the replicator's request rate (litestream's sync interval per database times the fleet's size) plus the socket round trip, and the budget to set from it; and, as an executable model, the transition table of revision 4 (both completion orders; `unissued`, `unrecorded`, `recovered-unknown`; discovery with zero pending; a failed discovery write under the pending-discovery latch; construction links and transitive inheritance across three fleets; the producer fence at a shutdown timeout; a kill at each boundary). |
| +- The design's admission is a commit of its own before permission (the `Start` transaction stays, so an attempt now costs two commits plus the settlement writes and the socket round trip to the helper): the gate's answer waits for it, and a commit that misses the budget refuses the request instead of sending anyway (the late commit is then marked unissued by the goroutine that awaited it). The cost is a second commit and a dependency: the request path's latency includes the ledger's commit, and a slow disk refuses requests rather than sending unrecorded ones. With the helper process the admission crosses a socket as well. The spike's deliverable: the ledger's commit latency distribution on a box's disk under the replicator's request rate (litestream's sync interval per database times the fleet's size) plus the socket round trip, and the budget to set from it; and, as an executable model, the transition table of revision 4 (both completion orders; `unissued`, `unrecorded`, `recovered-unknown`; discovery with zero pending; a failed discovery write under the pending-discovery latch; construction links and transitive inheritance across three fleets; the producer fence at a shutdown timeout; a kill at each boundary). |