amadan
rootfs apps: rev 2, answers Astra's builder check
Kind and hibernation fixed at creation, fleets only, always-on minted rather than converted, SQLite path kept in v1, overlapping deploys with worker handover. Egress allow widened to any nested instance on the customer's own boxes; stateless replicas recorded as wanted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 files changed,
+109
−21
docs/superpowers/specs/2026-10-24-rootfs-apps-design.md+92 −21docs/superpowers/specs/CODEX-FINDINGS-ROOTFS-APPS-R1.md+17 −0
CI cancelled — superseded by a newer push — run details
diff --git a/docs/superpowers/specs/2026-10-24-rootfs-apps-design.md b/docs/superpowers/specs/2026-10-24-rootfs-apps-design.md| index 4711c45b6..b3213c7e8 100644 |
| --- a/docs/superpowers/specs/2026-10-24-rootfs-apps-design.md |
| +++ b/docs/superpowers/specs/2026-10-24-rootfs-apps-design.md |
| @@ -1,8 +1,10 @@ |
| # Rootfs apps: run any Linux app on your own boxes, isolated like a CARLOS instance |
| **Date:** 2026-10-24 |
| -**Revision:** 1. Draft from a brainstorming session with Paul, 2026-10-24. |
| -Not yet reviewed. |
| +**Revision:** 2. Draft from a brainstorming session with Paul, 2026-10-24. |
| +Rev 2 answers Astra's builder check (`CODEX-FINDINGS-ROOTFS-APPS-R1.md`, |
| +six questions): §5.0 is new, §5.6 and §5.7 are narrowed, §5.2 settles |
| +deploy overlap. Marked **(B1-n)**. |
| **Discussion:** amadan carlos/platform (to be filed with this draft). |
| **Base:** origin/main 64a002c8. |
| @@ -185,6 +187,22 @@ else. |
| ## 5. The parts |
| +### 5.0 What fixes an app's kind (B1-2) |
| + |
| +- **Rootfs is a property of the app, fixed by its first release.** A |
| + release whose kind (binary or rootfs) differs from the app's is refused |
| + at ship, the way `doc_engine` with `ports` is refused today. Promotion |
| + and rollback therefore never cross kinds. Moving an app between kinds |
| + means a new app. |
| +- **Hibernation is fixed when the instance is created**, from the |
| + manifest of the release it is created with: always-on unless that |
| + manifest opts in. A later release that asks for the other setting is |
| + refused at ship. Flipping a live instance between the two needs its own |
| + transition, which v1 does not have. |
| +- The instance record carries the kind and the hibernation setting, so a |
| + box decides placement and launch from the record, not from whichever |
| + release happens to be staged. |
| + |
| ### 5.1 The rootfs artifact (ship and stage) |
| - `carlos ship --rootfs <source>`. `<source>` is either: |
| @@ -249,6 +267,16 @@ The manifest names processes: |
| rootfs row and the command came from the staged manifest the control |
| service resolved, never from tenant config. |
| - Binary apps are unchanged: no `processes`, same argv, same flags. |
| +- **Deploys overlap, as today (B1-5).** Rootfs apps follow the existing |
| + rollout policy (`internal/rollout/policy.go:23`, overlap by default): |
| + the new generation's `web` starts on its own socket, the edge switches |
| + when it is healthy, and the old `web` drains. Workers are handed over |
| + in the same order: the old generation's workers stop once the new |
| + `web` is healthy, then the new workers start. So two worker sets never |
| + run at once, and two `web` processes do only during the overlap. |
| + Schema migrations that run at boot (`db:prepare`) can run while the old |
| + release still serves, exactly as in a rolling Kubernetes deploy. Making |
| + them backward-compatible is the app's job, and the guide says so. |
| ### 5.3 The rootfs unit variant |
| @@ -325,11 +353,31 @@ hibernation. Always-on nested is new work: |
| the lease and the replica), whether or not the row hibernates. The |
| builder must not fall back to the box-wide litestream config for |
| nested rows. |
| -- The always-awake design (`implementation/always-awake-exec.md`) has |
| - worked through this lifecycle for flat exec instances: its |
| - lifecycle field, the refusals and the recovery contract. Reuse its |
| - conclusions where they apply. Its refusal of fleets and pools does not |
| - carry over: rootfs apps live only on fleets and pools (R1). |
| +- **Not a prerequisite: the always-awake design (B1-3).** |
| + `implementation/always-awake-exec.md` is about converting existing, |
| + initialized flat instances through maintenance-mediated activation. |
| + No rootfs instance exists yet, so there is nothing to convert. A rootfs |
| + instance is created always-on: the record is minted with its |
| + lifecycle (§5.0) and never changes it in v1. Its maintenance |
| + machinery, the activation operation and the migration preflight do not |
| + apply and are not built here. |
| +- What this design does take from that document: the meaning of |
| + always-on (no idle sleep, recovery without a visitor, stops only for |
| + stop, maintenance, lost ownership or shutdown), its list of mechanisms |
| + that do not count as residency (dummy requests, timers, a huge idle |
| + timeout), and its recovery cases (boot, crash, adoption, the |
| + four-start ceiling). Its refusal of fleets and pools does not carry |
| + over: rootfs apps live on fleets (R1, §5.7). |
| +- **Version floor.** A box running a release without rootfs support |
| + must never be handed a rootfs row. §5.7's rootfs-ready report is the |
| + floor: the console places a rootfs app only on boxes that report it, |
| + and an old box ignores the row (C2). |
| +- **A rootfs app with no local database (B1-4).** v1 keeps the SQLite |
| + path for every rootfs instance, used or not. An app whose data lives in |
| + MySQL ignores `CARLOS_DB`, and the platform replicates an empty file, |
| + which costs almost nothing. Declaring "no local database", and |
| + separating the lease from replication, belongs to the |
| + stateless-replicas design (§12), which needs it anyway. |
| - The plans-caps never-sleep refusals (plans 3b, not built) must not |
| refuse a rootfs app on its own boxes. A customer's own box costs |
| Carloku nothing idle. |
| @@ -337,8 +385,12 @@ hibernation. Always-on nested is new work: |
| ### 5.7 Where a rootfs row may run (R1) |
| - **Shared deployments** (carloku.com): only on a fleet box |
| - (`CARLOS_REGION=fleet/<name>`) or a box serving only non-shared pools |
| - (`CARLOS_POOLS` without `""`). |
| + (`CARLOS_REGION=fleet/<name>`). A fleet belongs to one account, and the |
| + reconciler already keeps fleet and platform records apart, so a fleet |
| + box carries one customer by construction. **Dedicated pools do not |
| + qualify in v1 (B1-1):** a box may serve several named pools, and |
| + nothing binds a pool to one account, so `CARLOS_POOLS` proves no |
| + ownership. Pools can join once a pool has an owning account. |
| - **Single-tenant deployments**: a deployment-level setting declares |
| that every box belongs to one customer, and then any box qualifies. |
| The builder names the setting; it is operator-only and lives with the |
| @@ -349,7 +401,8 @@ hibernation. Always-on nested is new work: |
| - Enforced in two places: the console refuses to create or place a |
| rootfs app where it may not run, and the box refuses to start one. |
| The box check is the one that counts; the console check is for a |
| - clear message. |
| + clear message. The box also requires the rootfs row's account to be |
| + the fleet's owning account. |
| - The `carlos status --layout` warning about flat neighbours (nested |
| isolation not in force) applies to a fleet box too. On a box owned by |
| one customer the neighbours are that customer's own, so this is a |
| @@ -383,12 +436,14 @@ runtime. |
| reachable today through the egress fence. |
| - **Private addresses** (an RDS or ElastiCache endpoint at `10.x` in the |
| customer's VPC): **blocked today.** The nested `IPAddressDeny=` covers |
| - RFC 1918, and no per-app egress allow exists on main. Moving an app |
| - whose database is private needs one. This design asks for it as a |
| - capability that is valid only for rootfs and binary rows on the |
| - customer's own boxes (R1): a list of CIDRs or hostnames the operator |
| + RFC 1918, and no per-app egress allow exists on main. Paul wants |
| + instances to be able to call out to databases (2026-10-24), and that |
| + covers binary instances as much as rootfs ones. This design asks for |
| + it as a capability for **any nested instance** on the customer's own |
| + boxes (R1's placement rule): a list of CIDRs or hostnames the operator |
| of that deployment grants, rendered as `IPAddressAllow=` on the |
| - instance's slice. The box's own addresses, IMDS and the box's |
| + instance's slice. On a shared box a private range is the platform's |
| + own network, so the grant refuses there. The box's own addresses, IMDS and the box's |
| loopback stay denied whatever the list says. |
| - **Uploads on Carloku**: `carlos store create`, no change. |
| - **An existing bucket** the app already uses: keys through `carlos env |
| @@ -478,6 +533,15 @@ resource IDs. Managed databases are the customer's choice of endpoint. |
| delete, and read-only by construction. |
| - Workers as co-spawn sidecars, not standalone members, because they |
| must share the instance's database. |
| +- Fleets only on shared deployments; dedicated pools wait for an owning |
| + account (B1-1). |
| +- Kind and hibernation fixed at creation; kind-changing releases refused |
| + at ship (B1-2). |
| +- Always-on rootfs instances are minted always-on and do not depend on |
| + the always-awake maintenance machinery (B1-3). |
| +- Every rootfs instance keeps the SQLite path in v1, used or not (B1-4). |
| +- Deploys overlap as today; workers are handed over after the new `web` |
| + is healthy; boot migrations are the app's job (B1-5). |
| - Always-on rows stay activator-owned and nested. |
| - Upload cap of 2 GiB for the rootfs part; worker cap of 4 per instance. |
| - Health bound of at most 120 s for rootfs apps. |
| @@ -488,10 +552,11 @@ resource IDs. Managed databases are the customer's choice of endpoint. |
| ## 12. Follow-ups (separate designs) |
| -- **Stateless replicas**: N copies of an app with no local state, on |
| - several boxes, behind the edge or a load balancer, with no lease and |
| - no litestream. Rootfs apps that declare no local database are the |
| - natural first users. Region HA's dormant load-balancer health answer |
| +- **Stateless replicas** (wanted, Paul 2026-10-24): N copies of an app |
| + with no local state, on several boxes, behind the edge or a load |
| + balancer, with no lease and no litestream. Two kinds of first user: |
| + rootfs apps that declare no local database, and existing binary apps |
| + that keep all their state in S3 with compare-and-swap. Region HA's dormant load-balancer health answer |
| and private forwarding are a starting point. |
| - **JIT opt-in**: a capability that drops `MemoryDenyWriteExecute` for |
| one app, on its own boxes only. |
| @@ -500,7 +565,13 @@ resource IDs. Managed databases are the customer's choice of endpoint. |
| ## 13. Open questions for Paul |
| +Astra's builder check (round 1) raised six questions. B1-1 to B1-5 are |
| +settled above as my calls. B1-6 is Q1. |
| + |
| - **Q1.** For the Rails app moving off Kubernetes (step 3), is failover |
| within minutes acceptable at first, or must it run on several boxes |
| - from day one? If the latter, stateless replicas (§12) move ahead of |
| - step 3. |
| + from day one? Paul (2026-10-24): stateless instances "make a ton of |
| + sense", and some apps already keep all their state in S3 with |
| + compare-and-swap. So the stateless-replicas design is wanted, for |
| + those apps as much as for rootfs ones. Its order relative to step 3 is |
| + still open. |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-ROOTFS-APPS-R1.md b/docs/superpowers/specs/CODEX-FINDINGS-ROOTFS-APPS-R1.md| new file mode 100644 |
| index 000000000..f000185f1 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-ROOTFS-APPS-R1.md |
| @@ -0,0 +1,17 @@ |
| +1. **What proves that every pool served by a box belongs to the same customer?** (§5.7) |
| + The proposed check accepts `CARLOS_POOLS` whenever it excludes the shared pool. Existing reconciliation checks pool membership, not common customer ownership; a box can serve several named pools. Is that configuration itself the operator’s declaration of exclusive ownership, or must the platform verify an ownership binding? The answer determines the admission checks needed to enforce R1. |
| + |
| +2. **Is rootfs execution fixed when an instance is created, or selected by its current release—and are binary↔rootfs transitions supported?** (§§4, 5.1–5.2, 5.6) |
| + Releases may contain both artifacts, but the design does not specify which an instance selects. Promotion or rollback could also cross execution types or change the manifest’s hibernation choice. The answer changes the persistent instance model, placement validation, transition protocol and mixed-version compatibility tests. |
| + |
| +3. **Which activation and recovery requirements from the always-awake design are prerequisites for rootfs v1?** (§5.6) |
| + [That document](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/design-rootfs-apps/implementation/always-awake-exec.md:160) requires enforced platform-version floors and describes maintenance-mediated activation for existing initialized instances, with additional runtime-contract requirements. Rootfs needs creation of new always-on instances. “Reuse its conclusions where they apply” leaves unclear whether those mechanisms must precede rootfs or whether a separate creation and recovery contract is intended. That substantially changes the dependency plan. |
| + |
| +4. **For an app using only MySQL or another external database, should CARLOS retain an unused SQLite replication path, or explicitly support no local database?** (§§5.2, 5.6, 6 step 3) |
| + The app may ignore `CARLOS_DB`, but the current [wake path always starts replication](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/design-rootfs-apps/internal/activator/activator.go:2589), and an empty database path falls back to the default. Explicitly supporting no local database requires separating lease ownership from replication, restore and checkpoint handling. The design does not specify that contract or how an app declares it. |
| + |
| +5. **Must rootfs v1 support overlapping deployments, or should every update stop the old web process and workers before starting the new release?** (§§5.3–5.6) |
| + The existing [deployment policy defaults to overlap](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/design-rootfs-apps/internal/rollout/policy.go:23). The design specifies instance and sidecar variants but does not settle generation overlap or worker replacement during deployment. This determines whether the plan needs concurrent image generations and worker handover, and whether boot-time `db:prepare` may run while the previous release still uses the database. |
| + |
| +6. **Does the Kubernetes migration require several serving boxes from its first launch, or is minutes-scale failover acceptable?** (§13 Q1) |
| + This remains explicitly unanswered at `a0b1a551`. It determines whether the separate stateless-replicas design must be implemented before step 3. |
| \ No newline at end of file |