| index c5568b85c..ce860fe47 100644 |
| --- a/docs/superpowers/specs/2026-10-21-instance-sizes-and-egress-design.md |
| +++ b/docs/superpowers/specs/2026-10-21-instance-sizes-and-egress-design.md |
| @@ -1,14 +1,15 @@ |
| # Instance sizes and egress allows |
| |
| Amadan #328 (an isolated instance reaching a private service its operator |
| -runs) and #329 (operator-set CPU and memory per instance). Revision 1, |
| -2026-10-21. |
| +runs) and #329 (operator-set CPU and memory per instance). Revision 2, |
| +2026-10-21. Revision 2 answers the builder check and Astra round 1 |
| +(`CODEX-FINDINGS-instance-sizes-egress-R1.md`); the disposition is §9. |
| |
| -Both are the same mechanism: **an operator-set property of an app or an |
| -instance, delivered to the box the way update settings already are, and |
| -written by the control service onto the instance's own slice** |
| -(`carlos-instances-<host>.slice`). #329 writes memory and CPU there, as |
| -`sliceTier` already does. #328 writes `IPAddressAllow=` there. |
| +Both are the same mechanism: **an operator-set policy for an app and its |
| +instances, kept in one operator-only object, delivered to the box, and |
| +written by root onto each instance's own slice** |
| +(`carlos-instances-<host>.slice`). #329 sets memory and CPU there, as |
| +`sliceTier` already does. #328 sets `IPAddressAllow=` there. |
| |
| ## 0. Rulings (Paul, 2026-10-20/21; do not reopen) |
| |
| @@ -27,7 +28,7 @@ written by the control service onto the instance's own slice** |
| way `EffectiveUpdates` resolves update settings. |
| 6. **The egress allow is operator-set only.** Never from a manifest, |
| never from an app's own credentials. (Sizes are operator-only too, but |
| - that is my call, §6.) |
| + that is my call, §8.) |
| 7. **Provider-neutral.** Nothing here names a cloud; a self-hosted box |
| with MinIO and plain hosts gets both features. |
| |
| @@ -47,9 +48,9 @@ written by the control service onto the instance's own slice** |
| |
| **Success:** an operator types one command per app to open one private |
| address for every instance of it, present and future, and one command to |
| -size an app or one instance; both take effect without a redeploy; and |
| -both can be seen in force on the running instance, read back from |
| -systemd rather than from intent. |
| +size an app or one instance; neither needs a redeploy; and both can be |
| +seen in force on the running instance, read back from systemd rather |
| +than from intent. |
| |
| ## 2. Goals, non-goals, anti-goals |
| |
| @@ -61,9 +62,11 @@ systemd rather than from intent. |
| - Every unit an isolated instance runs as (run, generation, seed, |
| sidecar, cmd, and the claim launch path) gets both, because all of |
| them sit in the per-host slice. |
| -- Granting, revoking and resizing reach running instances without a |
| - restart (§4.4 for the one exception), and survive a restart. |
| -- Visible: what systemd actually holds for each instance's slice, in |
| +- Grants and revokes reach running instances within seconds and are |
| + re-asserted until systemd's read-back matches. Growing a size reaches |
| + a running instance at once; shrinking memory waits for the next |
| + generation or start (§4.4). |
| +- Visible: what systemd actually holds for each instance, in |
| `carlos status --layout` and `carlos instances list`. |
| |
| **Non-goals** |
| @@ -74,190 +77,259 @@ systemd rather than from intent. |
| - Reservations or bin-packing. A size is a cap, not a reservation; see |
| §4.5 for what the console does check. |
| - Self-service sizes for app owners. That belongs with Carloku's plans |
| - and caps, which this must not undercut (§6). |
| + and caps, which this must not undercut (§8). |
| - A platform relay to private services, or making doc engines |
| fleet-internal: #335. |
| |
| **Anti-goals (failure even if it works)** |
| -- An app, a build or an instance able to grant itself an allow or a |
| - size. Every byte that reaches `systemctl` comes from operator-written |
| - state, never from a request or a manifest. |
| -- An allow that reaches another app's instances, or the probe, or any |
| - unit outside the app's own per-host slices. |
| +- An app, a build, an instance, or a compromised agent under host claims |
| + able to open an egress allow. Every address root writes comes from |
| + operator-written state (§3.4), never from a request, a manifest, or |
| + an agent-written file once claims are on. |
| +- An allow that reaches another app's instances, the probe, or any unit |
| + outside the app's own per-host slices. |
| - An allow that can open loopback, link-local, the metadata service, or |
| - any of the box's own addresses. |
| -- A revoke that leaves the address open on running instances. |
| -- A size change that OOM-kills a running instance. |
| + any address a fleet box holds. |
| +- A revoke that leaves the address open on a running instance without |
| + it being re-asserted and reported. |
| +- A size change that OOM-kills a running process. |
| - Reporting an allow or size as in force because it was requested. |
| |
| -## 3. Egress allow (#328) |
| +(Sizes carry the trust tiers carry today, not the egress allow's: §4.3.) |
| |
| -### 3.1 Where it lives |
| +## 3. Where the policy lives |
| |
| -On the app's instance manifest, `apps/<sqid>/<app>/instances.json` |
| -(`provision.Manifest`), as a new field beside `Updates` and |
| -`PrivateStorage`: |
| +One object per app, **`control/app-policy/<sqid>/<app>.json`**, written |
| +only by the console's operator doors (§5.1), compare-and-swap on its |
| +ETag: |
| |
| - "egress_allow": ["10.0.5.20", "fd00:5::20"] |
| + { |
| + "v": 1, |
| + "egress_allow": ["10.0.5.20", "fd00:5::20"], |
| + "default_size": "large", |
| + "sizes": { "shop-a.example.com": "xlarge" } |
| + } |
| |
| -**Precondition the builder must verify and record in the plan:** no |
| -credential an app, a build or an instance holds can write that key. |
| -Today the writers are the console, the boxes, and operators holding |
| -bucket credentials in bucket mode. If any app-held credential can write |
| -`apps/<sqid>/<app>/instances.json`, stop: the allow moves to a |
| -console-only key under `control/`, and the plan says so. |
| +Not on `instances.json` or the instance records: those are rewritten by |
| +member doors, and an older console's read-modify-write drops fields it |
| +does not know (`apiinstances.go:2471`, "recorded debt"), which would |
| +silently erase an operator's policy during a mixed-version roll |
| +(Astra R1-7). A separate object no older binary writes cannot be erased |
| +that way. |
| |
| -### 3.2 What is accepted |
| +- Deleting an instance removes its `sizes` entry (same door path as the |
| + delete). A stale entry is otherwise inert, but a host name can be |
| + reused, and a new instance must not inherit an old one's size. |
| +- **Precondition the builder must verify and record in the plan:** the |
| + key is under the store's control boundary (`hostops/mutation_boundary.go` |
| + and the bucket policy), so no app, build, instance or member |
| + credential can write it. If that does not hold for `control/` as a |
| + whole, the plan adds the prefix to the boundary before anything reads |
| + it. |
| |
| -Validated at the console door and again on the box (the box never trusts |
| -the bucket's copy more than the door's): |
| +## 4. Egress allow (#328) |
| + |
| +### 4.1 What is accepted |
| + |
| +Validated at the console door and again by root on the box: |
| |
| - A single IPv4 or IPv6 address. No CIDR, no zone, no port, no hostname. |
| IPv4-mapped IPv6 is unmapped first. |
| - **Must be inside the instance fence's deny list** (RFC 1918, CGNAT, |
| ULA, and the other private ranges of `IPAddressDeny=`). A public |
| address is already reachable; it is refused as "already reachable, no |
| - allow needed" so the list only ever holds real exceptions. |
| + allow needed". |
| - **Refused anywhere:** loopback, unspecified, all link-local (v4 and |
| v6), both metadata addresses (169.254.169.254, fd00:ec2::254), |
| multicast, 0.0.0.0/8, 192.0.0.0/24, 198.18.0.0/15, 240.0.0.0/4, the |
| NAT64 prefix 64:ff9b::/96 (it maps to any IPv4 address), fec0::/10. |
| -- **Refused at the console:** any address a fleet box publishes as its |
| - own private address (the edges file's `private_ip`, region HA 2a). |
| - Opening a box opens every listener on it. |
| -- **Refused on the box:** any of the box's own addresses (`isLocal`, |
| - `internal/instancectl/localaddrs.go`). The box drops that entry, keeps |
| - the rest, and reports it (§3.5). |
| +- **Refused if any fleet box holds it** (§4.2). |
| - At most 8 addresses per app. Duplicates collapse; the list is stored |
| sorted. |
| |
| -Reuse `ResolverAllowed` and `localPrefixes` where they fit; the deny-list |
| -ranges should come from one table shared with the template test |
| +Reuse `ResolverAllowed`'s refusals and `localPrefixes` where they fit. |
| +The deny-list ranges come from one table shared with the template test |
| (`internal/host/units_hardening_test.go`), so a range added to the fence |
| is automatically one an allow must fall inside. |
| |
| -### 3.3 Who can set it |
| - |
| -System operators only: the `apiSystemRoller` gate (owner of the ops |
| -account, or listed in `CARLOS_SYSTEM_OPERATORS`), with an empty service |
| -role set so no service credential reaches it. A new door, |
| -`PUT /api/cli/apps/{account}/{app}/egress`, compare-and-swap on the |
| -manifest's ETag. |
| - |
| -Every change is recorded with `s.audit` (account, app, old list, new |
| -list, operator). For this door the record is not best-effort: if the |
| -event cannot be written, the change is refused. |
| - |
| -### 3.4 How it reaches the instance |
| - |
| -1. The box's reconcile pass already reads each app's manifest. It |
| - converges `EgressAllow` into a new registry column on each of the |
| - app's route rows (`egress_allow`, the validated, sorted list), the way |
| - `SetUpdateSettings` converges update settings. |
| -2. The control service writes it. It never takes the list from a |
| - request: it looks the host's app up (`HostApp`) and reads the column |
| - from the registry itself, keeping "nothing from a request reaches |
| - systemctl as a string" (`server.go:33`). |
| -3. **When:** on every start path, unconditionally (not gated on a cold |
| - slice the way `sliceTier` is for generations and cmd units), and on a |
| - new `POST /egress?host=` call the agent makes after reconcile changes |
| - the column for a host with a populated slice. |
| -4. **What:** `systemctl set-property --runtime <slice> IPAddressAllow=` |
| - (reset) then `IPAddressAllow=<a>/32 <b>/128 ...`, the same reset-then- |
| - write shape as `allowResolvers`. An empty list writes only the reset. |
| -5. **Live:** the builder must prove on a real kernel (a cgroup test, the |
| - exception to "CI only") that a slice-level `set-property --runtime` |
| - change both opens and closes the address for a process already running |
| - in a child unit, and that the per-host slice's allow combines with the |
| - resolver allow on `carlos-instances.slice` rather than replacing it. |
| - If systemd does not apply a revoke live, a revoke restarts the host's |
| - units, and the plan says so. |
| - |
| -A move carries the allow for free: the destination's reconcile reads the |
| -same manifest. Re-isolation picks it up at the next start. A revert to |
| -flat leaves an unused slice; flat instances have no private-range fence, |
| -so there is nothing to carry. The column is cleared when the route row |
| -goes. |
| - |
| -### 3.5 Seeing it in force |
| - |
| -The control service reads the slice back (`systemctl show -p |
| -IPAddressAllow <slice>`) after every write and reports per host: |
| -`allowed` (what systemd holds, minus the resolver entries inherited from |
| -the parent) and `refused` (entries the box dropped, with the reason). |
| -That travels in the box's status snapshot, and: |
| +### 4.2 Fleet addresses |
| + |
| +Opening another box's address opens every listener on it, including the |
| +private paths boxes use to reach each other. The edges file's |
| +`private_ip` cannot answer this: it holds one IPv4 address and refuses |
| +IPv6 (`internal/edgesfile/private.go:15`) (Astra R1-5). |
| + |
| +- Each box publishes **every** non-loopback address it holds (from the |
| + same procfs sources as `localPrefixes`) in a new edges-file field, |
| + `addrs`, refreshed whenever it writes the file. |
| +- **Console:** refuses a grant naming any address in any box's `addrs`. |
| + If any box's edges file is missing, older than the 15-minute freshness |
| + bound `boxchoice` uses, or has no `addrs` field (an older box), the |
| + grant is refused, naming the box: the inventory cannot vouch for it. |
| +- **Root on each box:** before every write (§4.3), drops any entry that |
| + is one of its own addresses (`isLocal`) or appears in any fresh edges |
| + file's `addrs`, and reports the drop (§4.4). This catches a box that |
| + joins, or an address that moves, after the grant. |
| + |
| +### 4.3 How it reaches the instance, and who decides |
| + |
| +Root (the control service) writes it. It never takes addresses from a |
| +request (`server.go:33`). Its source follows the trust root already has |
| +for `HostApp`, the lookup that decides which build a nested generation |
| +may run (`validateStagedExe`): |
| + |
| +- **With host claims on**, `HostApp` is store-verified |
| + (`rootInstallationHostApp`: claim holder, installation, and the record |
| + root reads from the store). Root then reads |
| + `control/app-policy/<sqid>/<app>.json` from the store itself, for that |
| + verified account and app, and validates it (§4.1). The agent is |
| + treated as possibly compromised there, and nothing it wrote is read. |
| +- **Before claims**, `HostApp` reads the agent-written registry |
| + (`instanceRegistryHostApp`), so the agent already decides which build |
| + an isolated instance runs. The agent converges the policy into a |
| + registry column on each of the app's route rows; root reads that |
| + column and validates it. This adds no trust the agent does not already |
| + hold. **One extra rail:** a box whose layout is not wholly nested (any |
| + flat exec child, which runs as the agent's user and may be able to |
| + write `registry.db`) gets no allow written at all, and says so. |
| + |
| +**What root writes:** `systemctl set-property --runtime <slice> |
| +IPAddressAllow=` (reset) then `IPAddressAllow=<a>/32 <b>/128 ...`, the |
| +reset-then-write shape of `allowResolvers`. An empty list writes only |
| +the reset. |
| + |
| +**When:** desired state is re-asserted against read-back, not written |
| +once on change (Astra R1-4: a change-triggered write that fails, or an |
| +agent that crashes between the registry write and the call, would |
| +otherwise leave a revoked address open for good): |
| + |
| +- on every start path, unconditionally (run, gen, seed, sidecar, cmd, |
| + claim launch); |
| +- on a `POST /egress?host=` the agent sends after converging a change, |
| + for promptness; |
| +- and by root's own loop, every 30 seconds (the disk guard's cadence) |
| + and at control-service start: for every populated per-host slice, read |
| + back `IPAddressAllow`, compare with the desired list, rewrite on any |
| + difference, and report a host whose write keeps failing. |
| + |
| +**Must be proven before anything is built on it** (a kernel test, the |
| +exception to "CI only"): a slice-level `set-property --runtime` change |
| +both opens and closes the address for a process already running in a |
| +child unit, and the per-host allow combines with the resolver allow on |
| +`carlos-instances.slice` rather than replacing it. If systemd does not |
| +apply a revoke live, a revoke restarts the host's units, and the plan |
| +says so. |
| + |
| +A move carries the allow for free: the destination reads the same |
| +policy. Re-isolation picks it up at the next start. A revert to flat |
| +leaves an unused slice; flat instances have no private-range fence. |
| + |
| +### 4.4 Seeing it in force |
| + |
| +Root reads the slice back (`systemctl show -p IPAddressAllow <slice>`) |
| +after every write and in its loop, and reports per host: `allowed` |
| +(what systemd holds, minus the resolver entries inherited from the |
| +parent), `refused` (entries dropped, with the reason), and `failing` |
| +(desired and held differ after a write). That travels in the box's |
| +status snapshot, and: |
| |
| - `carlos status --layout` gains an `EGRESS` column: the read-back list, |
| - `-` for none, `!` plus the reason when the read-back disagrees with |
| - the column. |
| + `-` for none, `!` plus the reason when it disagrees with the policy. |
| - `carlos instances list` gains the same, terse. |
| - `carlos apps show` shows the configured list. |
| |
| -## 4. Sizes (#329) |
| +## 5. Sizes (#329) |
| |
| -### 4.1 The vocabulary |
| +### 5.1 The vocabulary |
| |
| One table in `internal/cgroup`, replacing the two `Tier*` variables as |
| the source of every number: |
| |
| -| size | memory max | high (90%) | CPU quota | CPU weight | IO weight | |
| -|---|---|---|---|---|---| |
| -| `small` | 256M | 230M | 50% | 50 | 50 | |
| -| `standard` | 512M | 460M | 100% | 100 | 100 | |
| -| `large` | 1G | 920M | 200% | 100 | 100 | |
| -| `xlarge` | 2G | 1840M | 400% | 200 | 200 | |
| +| size | memory max | CPU quota | CPU weight | IO weight | |
| +|---|---|---|---|---| |
| +| `small` | 256 MiB | 50% | 50 | 50 | |
| +| `standard` | 512 MiB | 100% | 100 | 100 | |
| +| `large` | 1 GiB | 200% | 100 | 100 | |
| +| `xlarge` | 2 GiB | 400% | 200 | 200 | |
| |
| -`standard` and `large` are exactly today's `TierDefault` and |
| -`TierChildren`, so an instance nobody sizes is unchanged byte for byte. |
| -(My call: the four names and their numbers. `small` halves the weights so |
| -a dormant instance yields under contention; `xlarge` doubles them so a |
| -busy one wins.) |
| +`MemoryHigh` stays `HighOf(max)` (`cgroup.go:128`, `max / 10 * 9`), the |
| +authoritative figure, so `standard` and `large` are byte-identical to |
| +today's `TierDefault` and `TierChildren` (Astra R1-9). (My call: the |
| +four names and their numbers. `small` halves the weights so a dormant |
| +instance yields under contention; `xlarge` doubles them so a busy one |
| +wins.) |
| |
| -### 4.2 Resolution |
| +### 5.2 Resolution |
| |
| - effective = instance override, else app default, else the build's tier |
| + effective = sizes[host], else default_size, else the build's tier |
| |
| where the build's tier is today's rule (`children` → `large`, else |
| `standard`). A size below the build's tier is allowed and the CLI warns |
| ("this build declares children; small may be OOM-killed"). The operator |
| owns the call; the platform never silently raises it. (My call.) |
| |
| -Stored as `Size` on `provision.Record` (override) and `DefaultSize` on |
| -`provision.Manifest`, resolved by an `EffectiveSize(rec, manifest, |
| -buildTier)` beside `EffectiveUpdates`, converged into a registry `size` |
| -column by reconcile. |
| +`EffectiveSize(policy, host, buildTier)` sits beside `EffectiveUpdates`; |
| +the agent converges it into a registry `size` column. |
| |
| -### 4.3 How it reaches the instance |
| +### 5.3 How it reaches the instance, and who decides |
| |
| - The box's tier source (`ExecBackend.Tier`/`TierOf`, `memtier.go`) |
| becomes the effective size for the host, so every caller of the |
| `memory_max` parameter (run, sidecar, seed, cmd, parent) and the claim |
| - snapshot (`claim_snapshot.go:229`) carry it. The claim bound-unit path's |
| - hard-coded `TierDefault` (`claimunit_inputs.go:74`) goes. |
| -- The control service's `parseRequest` accepts exactly the four sizes' |
| - `memory_max` values; `SharesOf` maps the value back to its row. |
| - `ValidParentLimit` accepts one size or the sum of any two (a swap |
| - overlap, including one across a resize). |
| + snapshot (`internal/activator/claim_snapshot.go:229`) carry it. The |
| + claim bound-unit path's hard-coded `TierDefault` |
| + (`claimunit_inputs.go:74`) goes. |
| +- **Every validator takes the shared table** (Astra R1-6): |
| + `parseRequest`, `claimLaunchTier` (`claimlaunch_root.go:163`), |
| + `claimIsolatedPaths` (`claimroleassembly.go:124`), `claimTier` |
| + (`claimunit.go:67`) and `ValidParentLimit` (one size, or the sum of |
| + any two: a swap overlap, including one across a resize). `SharesOf` |
| + maps a memory max back to its row. `small` and `xlarge` are tested |
| + through the complete claim launch. |
| +- **Trust:** a size travels from the agent, as the tier does today, and |
| + root checks only that it is one of the four. A compromised agent can |
| + therefore pick any of the four sizes for any instance: a resource |
| + question, not an isolation one, and the exposure the two tiers have |
| + now. Root does not work sizes out for itself. (My call; Astra R1-8 |
| + declined, §9.) |
| - Flat exec children get the memory half only, as today (`applyLimit` |
| writes no CPU). `carlos-app@` is untouched. |
| |
| -### 4.4 Live changes |
| - |
| -When reconcile changes a host's effective size and the slice is |
| -populated, the agent asks the control service to apply it |
| -(`POST /size?host=`), which calls `sliceTier` with the new size: |
| - |
| -- **Growing** applies at once. |
| -- **Shrinking** applies at once only if the slice's `memory.current` is |
| - below the new size's `MemoryHigh`. Otherwise it is held, retried on |
| - every reconcile, and reported as `pending` with the current usage. It |
| - is never forced: a forced shrink is an OOM kill (anti-goal). |
| -- A start always applies the effective size (cold slices already do; the |
| - populated-slice skips for generations and cmd units now apply the size |
| - when it differs from what the slice holds, under the same |
| - shrink rule). |
| - |
| -### 4.5 Capacity |
| +### 5.4 Changing a running instance's size |
| + |
| +A size is enforced in two places, and a change must account for both: |
| + |
| +- the per-host **slice** (`sliceTier`), and |
| +- a running **generation unit's own** `MemoryMax`/`MemoryHigh`/ |
| + `MemorySwapMax`, written into its drop-in at its start (`GenDropIn`, |
| + `gen.go:40`). Resizing only the slice would leave that cap binding |
| + (Astra R1-2). The run, seed, sidecar and cmd templates carry no |
| + memory cap of their own; the slice is theirs. |
| + |
| +**The activator applies live changes**, under the same per-host |
| +serialization as swaps, because it owns swaps (`swap.go:517` raises the |
| +parent before the successor starts). It never sends a resize while a |
| +parent is raised; a change that arrives mid-swap waits for the swap to |
| +retire and restore the slice. The populated-slice skips in `startGen` |
| +(`gen.go:116`) and the run path (`server.go:503`) stay exactly as they |
| +are: they protect the overlap allowance (Astra R1-1). |
| + |
| +- **Growing** (memory up, or any CPU or IO change): applied live by a |
| + `POST /size?host=` call. Root sets the slice first, then the serving |
| + generation unit (`set-property --runtime` on the unit), and reads both |
| + back. |
| +- **Shrinking memory never touches a running process** (Astra R1-3: a |
| + usage sample cannot make lowering `memory.max` safe, because the |
| + process can allocate between the sample and the write). The new size |
| + takes effect when a process starts under it: the next generation (a |
| + deploy's swap starts the successor at the effective size, and the |
| + overlap figure is old plus new), or the next cold start (a wake from |
| + hibernation). Until then it reports `pending`. An operator who needs |
| + it now restarts the instance. |
| +- **Read-back** reports the serving generation unit's `MemoryMax` when |
| + there is one, else the slice's, and `swap` while the parent is raised. |
| + |
| +### 5.5 Capacity |
| |
| Sizes are caps, not reservations; instances hibernate, and the sum of |
| caps on a box is routinely above its memory. So the console does not |
| @@ -265,7 +337,7 @@ refuse on a sum. It does: |
| |
| - **Refuse** a size whose memory max exceeds a quarter of the smallest |
| total memory among the boxes the app's instances can be placed on |
| - (`Headroom.MemTotal` from the edges file), saying the box and its |
| + (`Headroom.MemTotal` from the edges file), naming the box and its |
| total. (My call: one instance should never be able to take a box's |
| whole headroom.) |
| - **Report**, on every size change, the box's committed figure: the sum |
| @@ -275,58 +347,103 @@ refuse on a sum. It does: |
| |
| Placement (`boxchoice`) is unchanged. |
| |
| -### 4.6 Who can set it, and how it shows |
| - |
| -- System operators only, the same gate as §3.3, recorded with `s.audit`. |
| - Doors: `PUT /api/cli/apps/{account}/{app}/size` and |
| - `PUT /api/cli/instances/{host}/size` (`default` clears the override). |
| -- `carlos instances list` and `carlos status --layout` gain `SIZE`: the |
| - size the slice actually holds (read back from systemd's |
| - `MemoryMax`), with `>name` when a change is pending. |
| - |
| -## 5. Rollout and compatibility |
| - |
| -- An older box ignores the new manifest and record fields: no allow |
| - (fail closed) and today's tiers. An older console never writes them. |
| +### 5.6 Showing it |
| + |
| +`carlos instances list` and `carlos status --layout` gain `SIZE`: the |
| +size in force, read back as §5.4 says, with `>name` when a change is |
| +pending and `swap` during an overlap. `carlos apps show` shows the |
| +default and the overrides. |
| + |
| +## 6. Doors and roles |
| + |
| +- System operators only, both properties: the `apiSystemRoller` gate |
| + (owner of the ops account, or listed in `CARLOS_SYSTEM_OPERATORS`), |
| + with an empty service role set so no service credential reaches them. |
| +- `PUT /api/cli/apps/{account}/{app}/policy`, carrying any of |
| + `egress_allow`, `default_size`, and `sizes` changes, compare-and-swap |
| + on the policy object's ETag. The CLI's `apps set --egress-allow`, |
| + `apps set --size` and `instances set --size` all use it. |
| +- Every change is recorded with `s.audit` (account, app, old and new |
| + values, operator). For these doors the record is not best-effort: if |
| + the event cannot be written, the change is refused. |
| + |
| +## 7. Rollout and compatibility |
| + |
| +- An older box ignores the policy object: no allow (fail closed) and |
| + today's tiers. An older box also publishes no `addrs`, so the console |
| + refuses egress grants until every box is upgraded (§4.2). |
| +- An older console cannot erase the policy: it never writes |
| + `control/app-policy/`. |
| - Boxes before consoles, dev first (standing roll order). |
| -- The control service is root code: a census overlay and a #280 ruling |
| - before merge. |
| -- Nothing is on by default. With no field set, every unit, slice and |
| +- Root code changes: a census overlay and a #280 ruling before merge. |
| +- Nothing is on by default. With no policy object, every unit, slice and |
| probe line is byte-identical to today. |
| |
| -## 6. Calls I made (check these) |
| +## 8. Calls I made (check these) |
| |
| - **Sizes are operator-only**, not app-owner. #329 allowed either. On a |
| multi-tenant fleet an app owner who could pick `xlarge` would bypass |
| Carloku's plan caps. Self-service can come with plans and caps. |
| -- The four sizes and their numbers (§4.1). |
| -- Shrinking below the build's tier is allowed with a warning (§4.2). |
| -- The quarter-of-a-box refusal (§4.5). |
| -- Public addresses are refused as pointless; fleet boxes' private |
| - addresses are refused as dangerous (§3.2). |
| -- The audit write is mandatory for both doors. |
| - |
| -## 7. Left to the builder |
| - |
| -- Column names, the agent-to-control-service calls' exact shape, and the |
| - status snapshot fields. |
| +- The four sizes and their numbers (§5.1). |
| +- Shrinking below the build's tier is allowed with a warning (§5.2). |
| +- Memory shrinks wait for the next generation or start (§5.4). |
| +- The quarter-of-a-box refusal (§5.5). |
| +- Public addresses are refused as pointless; every fleet box address is |
| + refused as dangerous, and an incomplete inventory refuses the grant |
| + (§4.1, §4.2). |
| +- The audit write is mandatory for the policy door. |
| +- Before claims, root takes the allow from the agent-written registry, |
| + and writes none on a box that is not wholly nested; with claims, root |
| + reads the store itself (§4.3). |
| +- Sizes stay agent-supplied and root-checked against the set (§5.3). |
| + |
| +## 9. Review dispositions |
| + |
| +**Builder check (rev 1):** generation units carry their own memory cap, |
| +and the swap owns the slice during an overlap (now §5.4); root must not |
| +take an allow from the agent under claims (now §4.3); `claim_snapshot.go` |
| +does exist, at `internal/activator/`. |
| + |
| +**Astra round 1** (REJECT, 5 P1, 4 P2): |
| + |
| +| # | Finding | Disposition | |
| +|---|---|---| |
| +| 1 | Start-path resize destroys swap headroom | Accepted: skips kept, resizes serialized with swaps by the activator (§5.4) | |
| +| 2 | Slice resize misses the generation's own cap | Accepted (§5.4) | |
| +| 3 | Shrink check cannot guarantee no OOM | Accepted: memory shrinks never touch a running process (§5.4) | |
| +| 4 | A failed revoke can stay open | Accepted: root re-asserts against read-back on a loop and at start (§4.3) | |
| +| 5 | `private_ip` cannot cover every box address | Accepted: published `addrs` inventory, refusal on an incomplete one, root re-check (§4.2) | |
| +| 6 | Claim validators accept only two tiers | Accepted (§5.3) | |
| +| 7 | An older console can erase size policy | Accepted: policy moved to its own `control/` object (§3) | |
| +| 8 | Size authority comes from request values | Declined: sizes keep the tiers' existing trust; the anti-goal is now worded for the egress allow, which is the isolation boundary (§2, §5.3) | |
| +| 9 | Proposed highs are not byte-identical | Accepted: `HighOf` stays authoritative (§5.1) | |
| + |
| +## 10. Left to the builder |
| + |
| +- Column names, the call shapes, and the status snapshot fields. |
| - Whether `EGRESS`/`SIZE` fit `instances list` or only `status --layout` |
| and `instances show`, if the table gets too wide. |
| -- The order of the plan's tasks, provided the live-revoke kernel test |
| - (§3.4 step 5) comes before anything is built on it. |
| +- The order of the plan's tasks, provided the kernel test of §4.3 comes |
| + before anything is built on it. |
| |
| -## 8. Testing |
| +## 11. Testing |
| |
| -- Validation tables for every refused address class, both doors. |
| -- The kernel test of §3.4 step 5, with the resolver allow present. |
| +- Validation tables for every refused address class, at the door and in |
| + root. |
| +- The kernel test of §4.3, with the resolver allow present. |
| - A production-wired test per start path (run, gen, seed, sidecar, cmd, |
| claim) that the slice holds the allow and the size after start, through |
| the real control service and units, not fixtures. |
| - Revoke while running: the address is unreachable from the running |
| - process afterwards. |
| -- Shrink held while `memory.current` is above the new high; applied once |
| - it falls. |
| + process afterwards. A revoke whose first write fails is re-asserted by |
| + root's loop. |
| +- A grant naming another box's address is refused; one naming an address |
| + a box acquires later is dropped by root and reported. |
| +- Resize during a swap waits; growth reaches the serving generation's |
| + own cap; a memory shrink leaves the running process's limits alone and |
| + applies to the next generation. |
| +- `small` and `xlarge` through the complete claim launch. |
| - Another app's instance on the same box cannot reach the allowed |
| address; the probe cannot either. |
| -- An instance with nothing set: units and slice properties identical to |
| - main. |
| +- An instance with no policy: units and slice properties byte-identical |
| + to main. |