| index c48fdc073..c4c8a44aa 100644 |
| --- a/docs/superpowers/specs/2026-10-24-sidecar-isolation-design.md |
| +++ b/docs/superpowers/specs/2026-10-24-sidecar-isolation-design.md |
| @@ -1,6 +1,6 @@ |
| # Sidecar isolation: native sidecars on the nested tier, and no flat exec by default |
| |
| -**Date:** 2026-10-24 · **Status:** revision 3, approved to build by Paul |
| +**Date:** 2026-10-24 · **Status:** revision 4, approved to build by Paul |
| 2026-10-24 ("let's just do this and close the vulnerability and roll a |
| release when done"), for an independent |
| review (brief: `CODEX-REVIEW-BRIEF-SIDECAR-ISOLATION.md`) and then Paul · |
| @@ -22,6 +22,15 @@ not spawn history (D15); the create check uses the reconciler's own |
| placement set (D17); and fence retirement needs a complete process census |
| (D18). |
| |
| +**Revision 4** answers Astra's round 3 (`CODEX-FINDINGS-SIDECAR-ISOLATION-R3.md`, |
| +verdict reject: three P1s, two P2s), marked **(R3-n)**. Every finding was in |
| +an automatic transition that read flat-writable state or raced the agent, |
| +so revision 4 removes them: `CARLOS_NESTED_WORKERS` and `CARLOS_FLAT_EXEC` |
| +are explicit, per box, in root-owned `host.env`; the ratchet is gone; the |
| +agent's worker eligibility is fixed at its start; install refuses a |
| +pre-floor binary whenever `host.env` opts into workers (D11, D15, §8.2). |
| +Where revision 3's bullets below say otherwise, revision 4's sections win. |
| + |
| **Revision 3** answers Astra's round 2 (`CODEX-FINDINGS-SIDECAR-ISOLATION-R2.md`, |
| at 730ba093, verdict reject: three P1s, two P2s), marked **(R2-n)**, and |
| records Paul's rulings of 2026-10-24 (§11). It also cuts what ships to what |
| @@ -102,11 +111,11 @@ The decisions, for scanning (each is argued where it is made): |
| | D8 | The layout pass moves worker rows with their full extras inventory; a row whose database and its companions are absent on both sides moves without it (rev 2) | §3.8 | |
| | D9 | Every worker replicates through its lease only; the box-wide replicator and the restore drills exclude workers (rev 2) | §3.9 | |
| | D10 | A failed nested gate never reopens flat for a worker | §3.10 | |
| -| D11 | Workers move only once `carlos.prev` is itself worker-capable; install keeps both binaries at or above the floor (rev 3) | §3.11 | |
| +| D11 | Workers opt-in in `host.env`; eligibility fixed at agent start with both binaries at or above the floor; install refuses a pre-floor binary while opted in (rev 4) | §3.11 | |
| | D12 | `CARLOS_FLAT_EXEC=refuse\|report` plus an exact-host allowlist, read from `host.env` | §4.1 | |
| | D13 | The refusal sits beside the claim refusal in both spawn seams, with its own error | §4.2 | |
| | D14 | Refusals and a registry inventory of what `refuse` would stop are reported per host (rev 2) | §4.3 | |
| -| D15 | Unset `CARLOS_NESTED_WORKERS` is on; unset `CARLOS_FLAT_EXEC` is a per-box ratchet that turns to `refuse` once nothing would be refused (rev 3) | §4.4 | |
| +| D15 | Both settings explicit per box in root-owned `host.env`; no automatic default change; closure verified per box (rev 4) | §4.4 | |
| | D16 | A refused managed row nudges the layout pass for that host | §4.5 | |
| | D17 | (deferred) Creation is refused unless the reconciler's own placement set has a fresh box and every one reports nested workers runnable | §5 | |
| | D18 | (deferred) The fence retires per box on four observable conditions, the third a complete process census | §6 | |
| @@ -733,21 +742,24 @@ replication, hibernating or always-on, flat or nested: |
| first drill after the change is the evidence that those lineages are |
| sound after a period of two writers. One that fails is a finding for |
| that host, reported as any drill failure is. |
| -- **The running replicator, not the file, decides (R2-3).** |
| +- **The running replicator, not the file, decides (R2-3, R3-3).** |
| `/etc/litestream.yml` is regenerated by an operator (`carlos ops |
| - litestream`, `internal/ops/ops.go:149-175`, which prints `changed` for |
| - the caller to restart the replicator) and litestream reads it only at |
| - start (`carlos-litestream.service:16`). A config that no longer names a |
| - worker's database is therefore no evidence that the running daemon has |
| - let go of it. So the layout pass moves an always-on worker only when the |
| - box-wide replicator cannot be holding the file: `carlos-litestream` is |
| - not active, or it is active and its current invocation started after |
| - the config was last written (`ActiveEnterTimestamp` against the file's |
| - mtime, read through `systemctl show`) **and** that config does not name |
| - the file. Otherwise the move is deferred with the reason "the box-wide |
| - replicator still names this database: regenerate its config and restart |
| - it" (copy to review), and retried on every pass, so a failed restart is |
| - never taken for a done one. |
| + litestream`, `internal/ops/ops.go:149-175`) and litestream reads it only |
| + at start (`carlos-litestream.service:16`, `Restart=always` at `:32`). |
| + The layout pass moves an always-on worker only when **all** of these |
| + hold, read under the fence immediately before the first rename: |
| + - `/etc/litestream.yml` exists and does not name the worker's database |
| + (or does not exist at all), so any start from now on cannot open it; |
| + - `carlos-litestream`'s `ActiveState` is exactly `active` with |
| + `SubState` `running` and its `ExecMainStartTimestamp` later than the |
| + config's mtime, **or** exactly `inactive` (or `failed`) with no main |
| + PID and its cgroup empty; |
| + - no observation failed. |
| + Anything else (`activating`, `deactivating`, `reloading`, an unreadable |
| + property, a config naming the file) defers the move with the reason |
| + "the box-wide replicator may still hold this database" (copy to review), |
| + retried on every pass. Because the config already excludes the file, |
| + a restart or a pending start after the check opens a config without it. |
| - **The move is then path-blind** for every worker, as it already is for a |
| hibernating one: the lease lineage is keyed by host, so moving the file |
| moves nothing in the bucket. |
| @@ -774,64 +786,67 @@ recorded (`layoutleg.go:127-146`). For workers: |
| loudly"). An operator who needs a specific host up through a gate |
| failure adds it to the allowlist, knowingly (§4.1). |
| |
| -### 3.11 Decision D11: downgrade (rev 3: R1-2, R2-2) |
| +### 3.11 Decision D11: downgrade (rev 4: R1-2, R2-2, R3-1) |
| |
| A binary from before this change runs a nested worker row through the old |
| `StartWorkerStaged`: flat, with the env read from the flat directory and |
| no database path, and the row's own nested `DBPath` replicated by the old |
| lease code. Two copies, and unisolated. |
| |
| -Revision 1 guarded `systemApplier.install` only. Round 1 named the other |
| -ways an old binary reaches a row: the rollback after a failed gate |
| -(`restoreFiles`, `systemapply.go:1784-1800`), the recovery unit, which can |
| -exec `carlos.prev` itself (`carlos-system-recover.service:41`), and a move |
| -interrupted before its row CAS. Revision 2 answered with a third protocol |
| -floor; round 2 showed that the root guard's separately installed copy |
| -(`guardarm.go:148-166`, overriding the recovery unit's `ExecStart`) would |
| -not know it (R2-2), and on flagship no protocol floor is published at all. |
| - |
| -Revision 3 closes every path from one fact about them. **Each path that |
| -puts a binary in front of the rows picks one of two files: the current |
| -`carlos` or `carlos.prev`.** The recovery unit's shell runs one or the |
| -other (`:41`); the guard copy chooses `carlos.prev` when an apply's journal |
| -is present and the current binary otherwise (`guardexec.go:66-80`), and its |
| -floor route chooses between the same two (`:95-110`); `restoreFiles` puts |
| -`carlos.prev` back. And root records `carlos.prev`'s release beside it |
| -before every apply (`<optDir>/carlos.prev.version`, written in `notePrev`, |
| -`systemapply.go:818-830`) and keeps an older `carlos.prev` when the |
| -running build is not recorded known-good (`decidePrev`, `:792-815`). |
| - |
| -**Decision.** |
| - |
| -- **Workers move only when both files can run them.** The layout pass |
| - treats worker rows as eligible (D8), and the reconciler mints nested |
| - worker rows (D7), only on a box whose running build and whose |
| - `carlos.prev.version` both parse as releases at or above |
| - `release.WorkerLayoutFloor` (the first release carrying this spec). An |
| - unreadable or unparseable `carlos.prev.version`, or none, reads as below. |
| - Status says `workers: waiting for carlos.prev` (a closed word) while it |
| - holds. In practice a box qualifies after its second roll at or above the |
| - floor (§13). |
| -- **Once true, it stays true.** Neither file goes below the floor without |
| - an install, and install refuses that: |
| - - a candidate whose version parses below `WorkerLayoutFloor`, or whose |
| - unit set lacks `carlos-instance-worker@.service` (for a candidate whose |
| - version does not parse), is refused while any worker host is named, |
| - beside R9's guard and on its row test (below); |
| - - `decidePrev` never replaces `carlos.prev` with a binary below the |
| - floor, because the running build is at or above it, and keeps the |
| - existing `carlos.prev` otherwise. |
| - So the guard copy, whichever of the two it picks, picks a worker-capable |
| - binary. It needs no change, and neither does the recovery unit. |
| -- **The row test is R9's, not a row-only one:** a worker host is named when |
| - its row has `NestedPaths`, when `Classify` calls it fenced or in |
| - conflict, or when it has a journal, whether or not the row exists |
| - (`layoutReversalIncomplete`, `systemapply.go:1273-1310`). D8's status |
| - change lets the console see worker hosts too, and `layoutDowngradeBlocked` |
| - (`layoutguard.go:24-69`) gains `release.WorkerLayoutFloor`, judged on |
| - those hosts, and blocks a channel move below it while any is named. |
| -- `carlos system layout revert` reverts worker rows with the rest (they are |
| - in the pass); after it, install accepts an older candidate again. |
| +Every path that puts a binary in front of the rows picks one of two files, |
| +the current `carlos` or `carlos.prev`: the recovery unit's shell |
| +(`carlos-system-recover.service:41`), the root guard's copy |
| +(`guardexec.go:66-110`), and `restoreFiles`. Root records `carlos.prev`'s |
| +release beside it before every apply (`carlos.prev.version`, `notePrev`) |
| +and keeps an older `carlos.prev` when the running build is not recorded |
| +known-good (`decidePrev`, `systemapply.go:792-815`). |
| + |
| +Revision 3 let the agent's eligibility follow `carlos.prev.version` live. |
| +Round 3 showed the race (R3-1): a downgrade's own `decidePrev` copies N |
| +into `carlos.prev`, which made the running N agent eligible, and it could |
| +mint or move a worker after install's row check and before the candidate |
| +replaced it. |
| + |
| +**Decision (rev 4).** Nested worker state exists only on an operator's |
| +explicit, root-owned say-so, and install keys its refusal on that say-so, |
| +not on state the agent creates: |
| + |
| +- **`CARLOS_NESTED_WORKERS` is opt-in:** unset or `off` means workers mint |
| + and stay flat (today's behaviour); `on` is set in `host.env`, which is |
| + root-owned (`/etc/carlos`, inaccessible to every tenant tier). |
| +- **The agent's eligibility is fixed at its own start.** At start it reads |
| + `CARLOS_NESTED_WORKERS`, its own build, and `carlos.prev.version`. It |
| + mints nested workers (D7) and moves worker rows (D8) only if all three |
| + held then: the setting is `on`, and both builds parse at or above |
| + `release.WorkerLayoutFloor`. Nothing that changes while it runs (an |
| + install's `decidePrev` included) makes it eligible; only a restart does. |
| + Status says `waiting_for_prev` or `off` while it does not hold. |
| +- **Install refuses a candidate below the floor** (version below |
| + `WorkerLayoutFloor`, or, for an unparseable version, a unit set lacking |
| + `carlos-instance-worker@.service`) **when `host.env` says |
| + `CARLOS_NESTED_WORKERS=on`, or** when R9's row test names a worker host |
| + (`layoutReversalIncomplete`, `systemapply.go:1273-1310`: nested paths, |
| + fenced or conflict, or a journal). The first arm reads a root-owned |
| + file, so it does not depend on what the running agent has done, and it |
| + closes R3-1: an agent that could create nested state is one whose box |
| + says `on`, and on such a box install never activates a pre-floor binary. |
| + The refusal comes before the journal and names the setting and the |
| + hosts, with the way back: revert the layout, set `off`, restart the |
| + agent, then downgrade (copy to review). |
| +- **The two files stay worker-capable.** Once an agent has started |
| + eligible, both `carlos` and `carlos.prev` were at or above the floor at |
| + that moment; `decidePrev` only ever copies the running binary into |
| + `carlos.prev`, and install will not put a pre-floor binary in `carlos` |
| + while the setting is `on`. So the guard copy and the recovery unit, which |
| + pick one of the two, need no change. |
| +- **Residual, stated:** an operator who sets `off` in `host.env` and |
| + downgrades without restarting the agent first leaves a running agent |
| + that may still create nested state during the install. The copy for |
| + `off` says to restart; the row test catches any state created before the |
| + install's check. |
| +- The console's `layoutDowngradeBlocked` (`layoutguard.go:24-69`) gains |
| + `release.WorkerLayoutFloor` over observed worker hosts. |
| +- `carlos system layout revert` reverts worker rows with the rest. |
| |
| ### 3.12 Every consumer of the shape |
| |
| @@ -887,17 +902,7 @@ logged and taken as the safe reading): |
| refused (§4.2). |
| - `report`: every spawn proceeds as today, and each flat or foreign spawn |
| of a host not on the allowlist is recorded as a would-be refusal (§4.3). |
| -- unset: the ratchet of D15 (rev 3): `report` until the box's inventory is |
| - first empty under a nested intent, `refuse` from then on, with one |
| - exception. On a box whose **effective intent is |
| - flat by an operator's choice**, unset stays `report` in every release, |
| - because that box's every row is flat on purpose. Flat by choice is |
| - either `CARLOS_LAYOUT_DEFAULT=flat` with no recorded intent, or a |
| - recorded `flat` intent, whatever the default says (R1-6). Flat by |
| - fallback (the default is nested but the box is not ready, |
| - `defaultLayoutIntent`, `layoutleg.go:115-146`) is not a choice, and |
| - unset means `refuse` there (D10). An intent the agent cannot read counts |
| - as not chosen. |
| +- unset: `report` (rev 4, D15). An operator sets `refuse` per box. |
| - an unknown value: `refuse`, logged. An operator who typed the variable |
| meant to choose, and the reading that cannot expose a tenant is the one |
| to take. (This is the opposite of `CARLOS_LAYOUT_DEFAULT`'s unknown-value |
| @@ -980,45 +985,37 @@ these seams; nothing here changes them. |
| - **`carlos status --layout`** shows the policy and each refused or |
| allowlisted host beside its layout word. |
| |
| -### 4.4 Decision D15: sequencing, so no box loses a workload on upgrade (rev 3: R1-6, R2-4) |
| - |
| -Revisions 1 and 2 had a later release change both defaults behind a |
| -console guard. Round 2 showed the guard is judged when the channel moves, |
| -while each box applies the release later, so a workload created in between |
| -gets both changes unassessed (R2-4). Paul's ruling (§11 Q6) is that a |
| -default change must not stop what was running. Revision 3 makes the |
| -change happen on the box, at the one moment it is known to stop nothing. |
| - |
| -**Decision.** |
| - |
| -- **`CARLOS_NESTED_WORKERS` unset means on**, from release N, subject to |
| - D11's condition and the box's effective intent being nested. A worker |
| - that opens `CARLOS_SIDECAR_DB` and uses `CARLOS_AGENT_SOCKET` moves |
| - without noticing. Every native sidecar on our deployments today is the |
| - operator's, and §8.2's check covers them before release N rolls. From |
| - release N the documented contract is those two variables (D4). |
| -- **`CARLOS_FLAT_EXEC` unset is a ratchet.** It reads `report` until the |
| - first time the agent finds the box's inventory (D14) empty and the |
| - effective intent nested, and from then on `refuse`. The agent records |
| - that moment in `/data/.carlos/flat-exec-refuse` (written once, atomically, |
| - never removed by any code path) and reads it at start. So: |
| - - nothing that runs when the ratchet turns is refused by it, because the |
| - inventory was empty; |
| - - a row that arrives later meets `refuse`, which is the property: a new |
| - worker is minted nested, a new instance is nested by default, and a |
| - foreign row an operator adds by hand is refused and named in status |
| - (D14), with `carlos add` or the allowlist as the repair; |
| - - a box whose intent is flat by choice (§4.1) never turns. |
| - The file lives in the agent's data directory, which only `carlos` |
| - processes in the host namespace can write. Once the ratchet has turned |
| - there are none but the agent. |
| -- **An explicit setting always wins**: `refuse` or `report`, read at agent |
| - start as before, and the ratchet file is then ignored but not removed. |
| -- The inventory, the ratchet's state and its time are in status, so an |
| - operator can see which boxes have turned and why one has not. |
| - |
| -No console guard, no later release flipping a default, and no `host.env` |
| -edit on any box. |
| +### 4.4 Decision D15: sequencing, so no box loses a workload on upgrade (rev 4: R1-6, R2-4, R3-2, R3-4, R3-5) |
| + |
| +Revisions 1 to 3 changed a default automatically: behind a console guard |
| +(R2-4: the guard is judged before boxes activate), then with an on-box |
| +ratchet (R3-4: the inventory and the commit are not one transaction; R3-5: |
| +the ratchet file is under `/data`, which a flat tenant can write; R3-2: a |
| +flat tenant can make its own row foreign and hold the ratchet open |
| +forever). Every automatic transition so far has read state the flat tier |
| +can write. Revision 4 has none. |
| + |
| +**Decision.** Both settings are explicit, per box, in root-owned |
| +`host.env`, and nothing changes them but an operator: |
| + |
| +- **`CARLOS_FLAT_EXEC`:** unset means `report` in this release (no |
| + automatic change, no ratchet, no file under `/data`). An operator sets |
| + `refuse` on a box once its status shows what `refuse` would stop: the |
| + inventory of D14, read from the registry, dormant rows included. Anything |
| + still listed (a legitimate foreign row, or a row a flat tenant has made |
| + foreign, R3-2) is the operator's to resolve before or by setting |
| + `refuse`: repaired with `carlos add`, allowlisted knowingly, or simply |
| + refused. Under `refuse` a row a tenant made foreign is refused like any |
| + other, so a tenant cannot keep its flat process by editing its row. |
| +- **`CARLOS_NESTED_WORKERS`:** opt-in, as D11 says. |
| +- **Both take effect at agent start**, which an operator's `carlos system |
| + host-env write` plus the agent restart provides. |
| +- **Closure is verified, not assumed (R3-2):** B1 counts as closed on a |
| + box when its status shows `CARLOS_FLAT_EXEC=refuse` in force, an empty |
| + allowlist, and no worker row `flat` or `foreign`. §8.2's last step checks |
| + this on every box and lists any that do not. |
| +- Making `refuse` the unset default for new boxes is a later decision, not |
| + this release's. |
| |
| ### 4.5 Decision D16: the stragglers |
| |
| @@ -1172,7 +1169,7 @@ boxes and private services on the network. |
| | **The fence** (today, interim) | all of the above except metadata | reach metadata, for as long as it cannot leave its cgroup for an exempt one (§1.6; the fence's own hardening, recorded on a dev box by §9) | |
| | **Release N rolled** (§3 dark, `report`) | the same: nothing that runs has changed | (status now names every flat spawn) | |
| | **`CARLOS_NESTED_WORKERS=on`** on a box whose gate is ready | on that box, nothing from a new or moved sidecar: it sees its own directory only, no loopback, no private range, no metadata, no sibling pid. It can still get a flat process **if** the box mints flat (gate failing with no recorded intent, D10) or the row is foreign | read or write another tenant's files from its sidecar; call the agent socket for another host (D3) | |
| -| **`refuse` in force** (explicit, or the ratchet turned), empty allowlist | nothing flat on that box, by any of the four spawn paths, in any layout state: a flat or foreign row is refused, the gate failing included | start any process in the host namespace | |
| +| **`refuse` set** in `host.env`, empty allowlist | nothing flat on that box, by any of the four spawn paths, in any layout state: a flat or foreign row is refused, the gate failing included | start any process in the host namespace | |
| | **Console refusal on** (follow-up, D17) | (no change in power) | create a record the platform would place unisolated or never run; this is honesty, the boundary is the line above | |
| | **Fence retired** (follow-up, §6) | nothing new: there is no flat child left for the fence to filter | | |
| |
| @@ -1228,56 +1225,51 @@ source is unreachable"; another provider's adapter supplies its own line. |
| |
| --- |
| |
| -## 8. Rollout (rev 3) |
| +## 8. Rollout (rev 4) |
| |
| ### 8.1 Flags and defaults |
| |
| -| Setting | Where | Unset means | Explicit values | |
| +| Setting | Where | Unset means | Set | |
| |---|---|---|---| |
| -| `CARLOS_NESTED_WORKERS` | box `host.env` | on (D15), effective only under D11's condition and a nested intent | `on`, `off` (keeps workers minted and kept flat) | |
| -| `CARLOS_FLAT_EXEC` | box `host.env` | the ratchet: `report` until the box's inventory is first empty under a nested intent, `refuse` from then on (D15); `report` forever under a flat-by-choice intent | `refuse`, `report` | |
| +| `CARLOS_NESTED_WORKERS` | box `host.env` (root-owned) | off: workers mint and stay flat | `on`: eligible from the next agent start under D11 | |
| +| `CARLOS_FLAT_EXEC` | box `host.env` | `report` | `refuse`, `report` | |
| | `CARLOS_FLAT_EXEC_ALLOW` | box `host.env` | empty | exact hosts (D12) | |
| |
| -None of these needs setting on any of our boxes. Nothing is dark in |
| -release N except what D11 holds back until `carlos.prev` qualifies. |
| +Release N changes nothing that runs until an operator sets these. |
| |
| ### 8.2 Order |
| |
| The house order: dev boxes, then the dev console, then a check, then |
| -flagship boxes, then the flagship console. |
| +flagship boxes, then the flagship console. Setting `host.env` values on a |
| +box is a production change under AGENTS.md's scoped approval: the plan for |
| +it (accounts, regions, the SSM parameters `carlos system host-env write` |
| +reads, the boxes, the restart, verification and the way back) goes to Paul |
| +once, before step 4. |
| |
| 1. **Before the cut: the sidecar check.** Every native sidecar on dev and |
| flagship (all the operator's, #370) is checked for a computed database |
| path, a hard-coded agent socket, and loopback or private-address use |
| - (§3.4, §7.3). Each one that needs it ships a release reading |
| - `CARLOS_SIDECAR_DB` and `CARLOS_AGENT_SOCKET` first. Where an always-on |
| - worker's database is named in a box's `/etc/litestream.yml`, the |
| - operator regenerates it (`carlos ops litestream`, which D9 now excludes |
| - workers from) and restarts the replicator; until then D9's check holds |
| - that worker's move (§3.9). |
| -2. **Release N to dev boxes, then the dev console.** On each box: status |
| - shows the inventory; workers stay flat (`waiting for carlos.prev`); D9's |
| - exclusion and D19's fence are live. |
| -3. **Release N's patch (N.1) to dev boxes and the dev console.** Now |
| - `carlos.prev` is N, so D11's condition holds and dev boxes move their |
| - workers: hibernating ones at once, always-on ones within the awake |
| - grace. Verify from inside with a throwaway sidecar in a dev account |
| - whose release lists `/data` and `/run/carlos`, reads `/proc/1/status`, |
| - sends the probe's datagrams, opens `CARLOS_SIDECAR_DB`, calls |
| - `/v1/ready` and `/v1/next-due` over `CARLOS_AGENT_SOCKET`, and prints it |
| - to `carlos sidecars logs`; then sleeps, wakes on its timer and on |
| - `carlos sidecars wake`, crashes once (restarted), is killed by SIGTERM |
| - from inside once (a crash, restarted), and finishes once (parked). Then |
| - watch each dev box's inventory empty and its ratchet turn to `refuse`. |
| - A foreign row made with `carlos add --db` is then refused and named. |
| -4. **Flagship:** N to boxes region by region, console box last, then the |
| - flagship console; then N.1 the same way. Watch each box move its workers |
| - and turn its ratchet. A box whose ratchet has not turned within the |
| - awake grace after N.1 is named in status with what holds it. |
| -5. **The fence** stays until its retirement follow-up (§13). |
| - |
| -N.1 can be whatever patch is next on `main`; its only job here is to make |
| -N the box's `carlos.prev`. |
| + (§3.4, §7.3). Each that needs it ships a release reading |
| + `CARLOS_SIDECAR_DB` and `CARLOS_AGENT_SOCKET` first. Any box whose |
| + `/etc/litestream.yml` names an always-on worker's database has it |
| + regenerated (`carlos ops litestream`, which now excludes workers) and the |
| + replicator restarted (D9). |
| +2. **Release N to dev, then flagship** (boxes, then console). Status shows |
| + each box's inventory; nothing moves. |
| +3. **Release N.1 the same way.** Now `carlos.prev` is N on every box. |
| +4. **Per box, dev first: `CARLOS_NESTED_WORKERS=on`** and an agent restart. |
| + Workers move: hibernating ones at once, always-on ones within the awake |
| + grace. On dev, verify from inside with a throwaway sidecar (namespace |
| + listing, `/proc/1/status`, the probe's datagrams, `CARLOS_SIDECAR_DB`, |
| + `/v1/ready` and `/v1/next-due` over `CARLOS_AGENT_SOCKET`, logs; sleep, |
| + timer wake, `carlos sidecars wake`, a crash, a self-SIGTERM crash, a |
| + finish). |
| +5. **Per box, once its inventory is empty or resolved: |
| + `CARLOS_FLAT_EXEC=refuse`** and an agent restart. |
| +6. **Verify closure on every box (R3-2):** `refuse` in force, empty |
| + allowlist, no worker row `flat` or `foreign`. Any box that fails is |
| + named on #370 with what holds it. |
| +7. **The fence** stays until its retirement follow-up (§13). |
| |
| ### 8.3 Census |
| |
| @@ -1374,16 +1366,15 @@ Unit and package level: |
| excludes an always-on worker and the restore census includes it; the |
| shrink guard accepts a path leaving for that reason and names it, and |
| still refuses any other shrink. |
| -- D11 (rev 3): the pass and the reconciler treat workers as flat while |
| - `carlos.prev.version` is missing, unparseable or below the floor, and |
| - status says `waiting for carlos.prev`; install refuses a candidate below |
| +- D11 (rev 4): the pass and the reconciler treat workers as flat unless, |
| + at agent start, `CARLOS_NESTED_WORKERS=on` and both builds were at or |
| + above the floor (missing or unparseable `carlos.prev.version` reads as |
| + below); a change to `carlos.prev.version` while the agent runs changes |
| + nothing; status says `off` or `waiting_for_prev`; install refuses a |
| + pre-floor candidate whenever `host.env` says `on`; install refuses a candidate below |
| the floor, or one lacking the worker template, while a worker host is |
| named, including a host with only a journal. |
| -- D15's ratchet: unset reads `report` with a non-empty inventory; turns |
| - once on the first empty inventory under a nested intent, writes its file |
| - once, and stays `refuse` across an agent restart and a later non-empty |
| - inventory; never turns under a flat-by-choice intent; an explicit value |
| - wins and leaves the file. |
| +- D15 (rev 4): unset `CARLOS_FLAT_EXEC` is `report`; unset `CARLOS_NESTED_WORKERS` is off; both read once at agent start; no file under `/data` changes either. |
| - D19 replacement (R2-1): a hold `ReplaceWorker` whose worker start |
| answers uncertain restores nothing and stops the entry; an overlap one |
| falls back only after `RoleGone`. |
| @@ -1466,10 +1457,9 @@ packages' tests. |
| 5. **A third protocol floor?** Superseded: revision 3's D11 needs none |
| (R2-2). |
| 6. **How defaults reach existing boxes:** nothing that runs is stopped by a |
| - default change (recommendation taken). Revision 3 does it with the |
| - ratchet of D15 instead of keeping old defaults on established boxes, |
| - because the ratchet needs no per-box `host.env` change and turns every |
| - box once it is safe. |
| + default change (recommendation taken). Revision 4 changes no default at |
| + all: both settings are set per box by an operator in `host.env` (§8.2), |
| + under the scoped approval production changes need. |
| |
| ## 12. Review disposition |
| |
| @@ -1519,14 +1509,31 @@ judged reasonable. Citations spot-checked and held. |
| | R2-4 | P2: defaults change during the rollout window | **Accepted, by Paul's ruling (Q6).** No release flips a default behind a console guard; the flat-exec default is a per-box ratchet that turns only when nothing would be refused | D15, §4.4 | |
| | R2-5 | P2: D17 drops stale boxes that still run workers flat | **Deferred with D17.** The console check is not the boundary and is out of this release; the follow-up takes R2-5's fix (stale eligible boxes block) | §5, §13 | |
| |
| + |
| +### 12.3 Round 3 (Astra, `CODEX-FINDINGS-SIDECAR-ISOLATION-R3.md`, at 99017de5) |
| + |
| +Verdict: reject. R2-1 fixed; R2-5's deferral accepted. Every round-3 |
| +finding was in an automatic transition that read state the flat tier can |
| +write, or that raced the agent. Revision 4 removes the transitions |
| +(`refuse-up-front-over-tolerating`). |
| + |
| +| Finding | | Disposition | Where | |
| +|---|---|---|---| |
| +| R3-1 | P1: install's `decidePrev` makes the agent eligible after the row check | **Accepted.** Eligibility is fixed at agent start; workers are opt-in in root-owned `host.env`, and install refuses a pre-floor candidate whenever `host.env` says `on` | D11, §3.11 | |
| +| R3-2 | P1: a flat tenant makes its row foreign and holds the ratchet open | **Accepted.** No ratchet; `refuse` is an operator's explicit setting, under which a foreign row is refused; closure is verified per box | D15, §4.4; §8.2 step 6 | |
| +| R3-3 | P1: "not active" is not proof the replicator let go | **Accepted.** Config must exclude the file in every case; only exactly `active/running` started after the config, or `inactive`/`failed` with no PID and an empty cgroup, passes; transitional or unreadable defers | D9, §3.9 | |
| +| R3-4 | P2: the ratchet overtakes a flat worker | **Accepted, by removal.** No ratchet | D15 | |
| +| R3-5 | P2: a flat tenant forges the ratchet file | **Accepted, by removal.** No file under `/data` decides policy; both settings are in root-owned `host.env` | D15 | |
| + |
| --- |
| |
| ## 13. What release N carries, and what follows |
| |
| -**Release N (and its patch N.1, §8.2):** D1 to D14, D16 and D19, with D11 |
| -and D15 as in revision 3. That closes B1: a native sidecar runs nested on |
| -every box whose `carlos.prev` qualifies, and once a box's ratchet turns, |
| -nothing on it runs in the host namespace unless an operator allowlists it. |
| +**Release N (and its patch N.1, §8.2):** D1 to D14, D16 and D19, with D9, |
| +D11 and D15 as in revision 4. That closes B1: a native sidecar runs nested on |
| +every box whose `carlos.prev` qualifies, and once an operator sets |
| +`refuse` on a box (§8.2 steps 5 and 6), nothing on it runs in the host |
| +namespace unless an operator allowlists it. |
| |
| **Follow-ups, each its own branch, filed as amadan discussions:** |
| |