| new file mode 100644 |
| index 000000000..4711c45b6 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-24-rootfs-apps-design.md |
| @@ -0,0 +1,506 @@ |
| +# Rootfs apps: run any Linux app on your own boxes, isolated like a CARLOS instance |
| + |
| +**Date:** 2026-10-24 |
| +**Revision:** 1. Draft from a brainstorming session with Paul, 2026-10-24. |
| +Not yet reviewed. |
| +**Discussion:** amadan carlos/platform (to be filed with this draft). |
| +**Base:** origin/main 64a002c8. |
| + |
| +--- |
| + |
| +## 1. Intent |
| + |
| +A tenant wants to move a Rails app off Kubernetes. Other people will want |
| +to bring Rails, Laravel, Django or Node apps too. Today CARLOS runs one |
| +kind of process: a single Go-style binary launched as |
| +`<binary> --socket <path> --db <path>`. Anything with an interpreter, |
| +gems, native libraries or more than one process has nowhere to go. |
| + |
| +The shape Paul asked for is Elastic Beanstalk's: **bundle the app into an |
| +artifact, push it, and it runs isolated.** The bundle is a read-only root |
| +filesystem built from an OCI image on the developer's machine. The box |
| +runs it as a nested instance under systemd, with the same namespace, |
| +cgroup and fences every nested instance gets. No Docker on the box, no |
| +registry, no build service. |
| + |
| +Why now: the nested layout (#126) is the default on boxes that are ready, |
| +so the isolation a rootfs app would rely on exists. The missing parts are |
| +the artifact, the launch contract and a few lifecycle pieces. |
| + |
| +## 2. Goals, non-goals, anti-goals |
| + |
| +### Goals |
| + |
| +- G1. A developer builds an arm64 image locally (Docker, Podman, |
| + buildpacks, anything that makes an OCI image or a directory tree) and |
| + ships it with `carlos ship`. It runs on their boxes with no further |
| + steps. |
| +- G2. A rootfs app gets the nested isolation a CARLOS binary gets: its |
| + own directory, no view of other apps' builds or data, no IMDS, the |
| + same memory tiers, the same egress fence. |
| +- G3. One image can declare more than one process: a web process that |
| + serves requests, and worker processes (Solid Queue, Sidekiq, a queue |
| + consumer) that share the instance's directory and environment. |
| +- G4. Rootfs apps are **always-on by default**. Hibernation is an opt-in. |
| +- G5. A developer can check, free, on their own machine, that an image |
| + meets the contract before they ship it. |
| +- G6. Nothing about the feature is Rails-specific or AWS-specific. |
| + |
| +### Non-goals (this design) |
| + |
| +- **Multiple copies of one app across boxes** (stateless replicas, load |
| + balancing, active-active HA). An instance still runs on exactly one |
| + box. HA is failover, from region HA once it is activated. A follow-up |
| + design covers stateless replicas (§12). Paul has not yet said whether |
| + the first real migration needs it on day one (§13, Q1). |
| +- **More than one SQLite file per instance.** Replication covers exactly |
| + one database per route today (`internal/host/caps_pass.go:302`). Rails |
| + 8's Solid Queue, Cache and Cable can share the primary database, and |
| + the sample app does that (§6.1). |
| +- **Building on the platform.** The platform never builds, never pulls |
| + from a registry and never runs Docker. |
| +- **JIT runtimes.** `MemoryDenyWriteExecute=yes` stays. Ruby's YJIT, |
| + PHP's opcache JIT, V8 and the JVM's JIT need writable and executable |
| + memory. Ruby, PHP and Python run without a JIT. A capability that |
| + drops this one line is deferred (§12). |
| +- **Free hosted trials.** Testing is free on the developer's machine |
| + (G5). Carloku does not run trial boxes. |
| + |
| +### Anti-goals (a failure even if it technically works) |
| + |
| +- A rootfs app running on a **shared** box. Arbitrary images are the |
| + least-proven thing the platform would run, and on a shared box an |
| + escape reaches other customers. |
| +- A rootfs app running **flat**. A flat instance shares the agent's |
| + user and its view of `/data`. A rootfs row must refuse to start on a |
| + flat box rather than run unisolated. |
| +- A second isolation stack (dockerd, containerd, podman) on the box, or |
| + any process the agent's cgroup and the isolation probe do not cover. |
| +- An old box that does not understand rootfs staging the image and |
| + trying to exec it as a binary. |
| + |
| +## 2.1 Rulings (Paul, 2026-10-24; do not reopen) |
| + |
| +- **R1. Own boxes only.** Rootfs apps run only on boxes the customer |
| + owns: a customer fleet or a dedicated pool on a shared deployment, or |
| + any box of a single-tenant deployment. |
| +- **R2. Never builds.** Builds are local, then pushed to the platform. |
| + The bucket stays the only control plane. |
| +- **R3. Free on their machine.** The free test path is a local check. |
| + No hosted trial boxes. |
| +- **R4. Runtime only.** Postgres, MySQL, Redis and uploads are served by |
| + whatever already serves them (managed AWS services in the same region, |
| + or Carloku's own `carlos store`). The platform runs the app's |
| + processes. |
| +- **R5. Always-on by default.** "Hibernating Rails apps suck anyway with |
| + that long wake-from-sleep." Hibernation stays available as an opt-in. |
| +- **R6. Language-neutral.** The design names Rails as the first case and |
| + must work for PHP (Laravel, FrankenPHP), Django and similar. |
| + |
| +## 3. What exists today, and what this builds on |
| + |
| +All citations are origin/main. |
| + |
| +- **Nested instances** (#126, plans 1 to 5; default layout since |
| + 00c20002). `internal/host/units/carlos-instance@.service`: |
| + - It runs `ExecStart=/opt/carlos/carlos instance-exec run %I`. The |
| + trampoline reads the launch file and execs the app as its child, |
| + staying as the namespace's init. |
| + - The namespace block: `ProtectSystem=strict`, empty `/data`, |
| + `/run/carlos` and `/opt/carlos`, `PrivatePIDs`, `PrivateIPC`, |
| + `CapabilityBoundingSet=` empty, `NoNewPrivileges`, |
| + `MemoryDenyWriteExecute`, and an `IPAddressDeny=` that blocks |
| + loopback, RFC 1918, CGNAT, link-local and IMDS (line 142). |
| + - Writable paths are the instance's two directories only (line 53). |
| + - The app's own build is bound read-only at `/opt/carlos/app` and |
| + `/opt/carlos/live/<host>` (line 69). |
| + - The host's `/usr` is visible read-only. The app sees the box's OS, |
| + which has no Ruby, Python or PHP. |
| +- **The launch contract.** `internal/activator/backend_exec.go:2436` |
| + builds `[]string{liveLink(host), "--socket", sock, "--db", dbPath}`. |
| + The child gets the instance's `.env` and `.config.env` plus |
| + `CARLOS_CONTROL_SOCKET`, `CARLOS_PURGE`, `CARLOS_REGION` and |
| + `CARLOS_HOST` (`agentOwnedVars`, :2310). The trampoline requires |
| + `argv[0]` to sit directly under `/opt/carlos/live` and be named for the |
| + host (`internal/instanceexec/exec.go`). Health: `GET /` on the socket |
| + must answer below 500 within 30 s (`spawnHealthTimeout`, :1426). |
| +- **The isolation probe** (`carlos-instance-probe.service`). It shares |
| + the namespace block byte for byte with the instance, seed and sidecar |
| + templates (`internal/host/units_isolation_test.go`), so a property it |
| + proves is a property a tenant gets. |
| +- **Co-spawn sidecars** (legacy, live). `internal/activator/activator.go:17`. |
| + A sidecar starts after the instance is healthy and stops first at |
| + sleep. It runs `liveLink(host) sidecar run` with no `--db` or |
| + `--socket` (`backend_exec.go:3188`), and its env is only |
| + `<host>-sidecar.env` (`prepare.go:94`). Nothing on the platform writes |
| + that file; it exists only if someone places it on the box. |
| +- **Standalone sidecar members** (671d3d11). Each has its own row, |
| + identity and directory, so it cannot share an instance's database. |
| +- **Hibernation and always-on.** Nested rows require `Hibernate` |
| + (`internal/activator/layout.go:63`, `internal/provision/reconcile.go:459`). |
| + An always-on exec instance stays flat and is replicated by the |
| + box-wide litestream config, not the activator. Members cannot ask for |
| + a never-sleeping routed instance (`apiinstances.go:~1026`). The |
| + always-awake design (`implementation/always-awake-exec.md`, merged as a |
| + document, not built) proposes a `lifecycle: always-awake-exec-v1` |
| + record field and refuses fleets and pools. |
| +- **Replication.** One database per route, `DBPath` single-valued. For a |
| + nested row it is `/data/instances/<iid>/<host>.db`. |
| +- **Customer fleets.** `carlos fleets create|add-box` (`fleets.go`). |
| + Fleet boxes are outbound-only agents with `CARLOS_REGION=fleet/<name>`. |
| + The reconciler keeps fleet and platform records apart |
| + (`internal/provision/reconcile.go:~398`). Plans 3a makes fleets |
| + paid-only (`internal/console/fleets.go:175`). |
| +- **Dedicated pools.** `Route.Pool`, served by `CARLOS_POOLS`. Pools are |
| + set at create time by operator flags only. |
| +- **Releases.** `Manifest.Artifacts` is keyed by platform |
| + (`linux-arm64`) or `site.tar.gz` (`internal/release/release.go:80`). |
| + The adopt pass stages only the artifact matching the box's |
| + `GOOS-GOARCH` (`adopt.go:31`) into `/opt/carlos/versions/<ver>/...`, |
| + sha256-checked. The ship door caps an upload at 256 MiB across all |
| + parts (`internal/console/apicli.go:1647`). Old versions are never |
| + reaped (#122). |
| +- **Object storage.** `carlos store create` provisions a bucket and IAM |
| + user and delivers credentials as env with a configurable prefix |
| + (`internal/console/instanceprivatestore.go:165`). Rails Active Storage |
| + and Laravel's S3 disk read that env directly. |
| + |
| +## 4. The shape, in one paragraph |
| + |
| +A release may carry a **rootfs artifact** instead of, or beside, a |
| +binary: one read-only filesystem image (erofs or squashfs) built by |
| +`carlos ship` from a local OCI image or directory, plus a **process |
| +declaration** in the manifest naming the web command and any worker |
| +commands. A box stages the image like a binary. The control service |
| +starts the instance from a **rootfs variant** of the nested template, |
| +which mounts the image as the unit's root (`RootImage=`), binds the |
| +platform binary and the instance's two directories into it, and keeps |
| +every line of the namespace block that still means something inside an |
| +image. The web process gets its socket and database paths as |
| +environment variables, not flags. Declared workers run as co-spawned |
| +sidecars from the same image with the same environment. The instance is |
| +always-on unless it opts into hibernation. A rootfs row starts only on |
| +a box that is nested and owned by one customer, and refuses everywhere |
| +else. |
| + |
| +## 5. The parts |
| + |
| +### 5.1 The rootfs artifact (ship and stage) |
| + |
| +- `carlos ship --rootfs <source>`. `<source>` is either: |
| + - an OCI image reference in the local image store or an OCI layout |
| + directory (`docker save`, `podman save`, buildpacks' output), or |
| + - a plain directory tree. |
| +- The CLI flattens the layers locally, checks the architecture is |
| + arm64 (refuse otherwise, naming the `buildx` fix), and writes one |
| + read-only image file. It adds the empty mount points the unit binds |
| + over (`/data`, `/run/carlos`, `/opt/carlos`, `/tmp`, `/dev`, `/proc`, |
| + `/sys`) if the image lacks them. |
| +- The manifest gains a rootfs artifact entry (file, sha256, size) and |
| + the process declaration (§5.2). Both are additive to schema 1 in the |
| + way `Capabilities` was. |
| +- **Old boxes must never stage it as a binary.** The artifact's key must |
| + not match `adoptPlatform`'s `GOOS-GOARCH` lookup on a release that |
| + predates this feature, and a release carrying only a rootfs artifact |
| + must be skipped by an old box exactly as a box with no matching build |
| + skips a release today. The builder picks the key; the test is that a |
| + previous release's adopt pass, run against a rootfs-only manifest, |
| + stages nothing. |
| +- **Upload size.** 256 MiB is too small: Rails images run 300 MB to |
| + 1 GB. Rootfs uploads keep streaming through the console (the `carlos |
| + run` ruling F8: never presigned) with a larger cap for the rootfs part. |
| + My call: 2 GiB. The box stages it streaming to disk, as #269 does for |
| + binaries. |
| +- **Disk.** Before rootfs apps ship to anyone, old versions must be |
| + reaped (#122) at least for rootfs artifacts, or a few deploys fill a |
| + box. The builder can scope the reaper to rootfs releases if the |
| + general one is not ready. |
| +- Image format: erofs or squashfs, whichever the boxes' kernel and |
| + systemd mount without extra packages. Builder's call; state the |
| + choice in the plan. |
| + |
| +### 5.2 The process declaration and the launch contract |
| + |
| +The manifest names processes: |
| + |
| +```json |
| +"processes": { |
| + "web": {"command": ["bundle", "exec", "puma", "-C", "config/puma.rb"]}, |
| + "worker": {"command": ["bin/jobs"]} |
| +} |
| +``` |
| + |
| +- Exactly one `web`, required. Zero or more others, names matching a |
| + short identifier pattern. Commands are argv arrays, absolute or |
| + resolved against the image's `PATH`. Optional `env` (plain values, |
| + no secrets) and `working_dir` per process. |
| +- The web process gets, in its environment: |
| + - `CARLOS_SOCKET`: the unix socket path it must bind and serve HTTP on. |
| + - `CARLOS_DB`: the path of the one SQLite file the platform |
| + replicates. An app that keeps its data elsewhere ignores it. |
| + - Everything a binary child gets today (`.env`, `.config.env`, the |
| + agent-owned `CARLOS_*` vars, store credentials). |
| +- Health is unchanged: `GET /` on `CARLOS_SOCKET` below 500. 30 s may be |
| + short for a Rails boot on a small box. The builder may give rootfs |
| + apps a longer bound, at most 120 s, and must say where it lives. |
| +- The trampoline gains a rootfs launch shape. Its argv check (`argv[0]` |
| + under `/opt/carlos/live`) does not apply, because the command lives in |
| + the image. Instead it accepts a rootfs launch only when the row is a |
| + rootfs row and the command came from the staged manifest the control |
| + service resolved, never from tenant config. |
| +- Binary apps are unchanged: no `processes`, same argv, same flags. |
| + |
| +### 5.3 The rootfs unit variant |
| + |
| +A rootfs instance runs from a variant of `carlos-instance@.service` (and |
| +of the sidecar and seed templates if seeds are supported; see §5.5). |
| + |
| +- `RootImage=` points at the staged image, through the control |
| + service's per-start `exe/<iid>` links (§3, line 69), so one instance |
| + sees its own image and no other. |
| +- Bound into that root: `/opt/carlos/carlos` (the trampoline, read-only), |
| + `/data/instances/%I` and `/run/carlos/instances/%I` (read-write, as |
| + today), the box's resolver config, and `/tmp` as a private tmpfs. |
| +- The namespace block keeps every property the probe proves today. Some |
| + lines change meaning inside an image (`ProtectSystem=` now protects |
| + the image, `TemporaryFileSystem=/data:ro` and friends now cover |
| + directories the image owns). The builder must restate each line of |
| + the block for the rootfs variant, keep the ones that still bind, and |
| + replace any that become no-ops with something that proves the same |
| + property. |
| +- **The probe gets a rootfs mode.** A probe image with the same variant |
| + checks the same properties from inside an image. A box reports rootfs |
| + readiness only after the rootfs probe passes, the way `/ready` |
| + depends on the probe for nested today. |
| +- Same slice, tier, `OOMPolicy`, `TasksMax`, log socket, `KillMode`, |
| + stop budget and `IPAddressDeny=` as the binary template. |
| + |
| +This is the part where review time goes. The original nested design |
| +took eleven review rounds; expect the variant to need a real security |
| +review of its own. |
| + |
| +### 5.4 Worker processes |
| + |
| +- Each non-web process runs as a co-spawn sidecar of the instance, from |
| + the rootfs sidecar variant, with the same image, directory, slice and |
| + environment as `web` (including `CARLOS_DB`). Today's sidecar gets |
| + only `<host>-sidecar.env` and no database path; rootfs workers get |
| + the instance's environment instead. |
| +- Lifecycle as today's co-spawn: start after `web` is healthy, stop |
| + first. With an always-on instance (§5.6) that means workers run all |
| + the time. |
| +- Several workers per instance are allowed. The builder bounds the |
| + count (my call: 4) and defines what one crashing worker does to its |
| + siblings and to `web`. My call: a crashed worker is restarted with |
| + backoff and never takes `web` down; a crash loop is reported in |
| + `carlos logs` and status. |
| +- Two processes writing one SQLite file is fine on one box in WAL mode. |
| + Litestream replicates the one file regardless of writer. |
| + |
| +### 5.5 Seeds, run, moves |
| + |
| +- **Seeds** (first-boot commands such as `rails db:prepare`): the |
| + existing seed mechanism runs a command once before first serve. A |
| + rootfs app may declare a `seed` process. Builder's call whether this |
| + lands in v1; the sample app can run `db:prepare` from its own boot. |
| +- **`carlos run`** (one-off commands, `carlos-instance-cmd@`): a rootfs |
| + variant makes `carlos run rails console` and `rails db:migrate` work. |
| + Desirable, not required for v1. |
| +- **Moves.** A rootfs app on a fleet box does not move between regions |
| + in v1. Moves refuse a rootfs row. A move would also break an app that |
| + relies on a same-region database (R4). |
| + |
| +### 5.6 Always-on nested instances |
| + |
| +R5 makes rootfs apps always-on by default, and today nested requires |
| +hibernation. Always-on nested is new work: |
| + |
| +- A rootfs row carries `hibernate: false` unless the manifest opts into |
| + hibernation. Hibernating rootfs rows behave like any nested instance. |
| +- An always-on nested instance is started at boot and after adoption, |
| + restarted after a crash, and stopped only by stop, maintenance, lost |
| + ownership or shutdown. "Always-on" means no idle sleep and no waiting |
| + for a visitor to recover it, not zero downtime. |
| +- Replication stays with the activator for nested rows (one owner of |
| + the lease and the replica), whether or not the row hibernates. The |
| + builder must not fall back to the box-wide litestream config for |
| + nested rows. |
| +- The always-awake design (`implementation/always-awake-exec.md`) has |
| + worked through this lifecycle for flat exec instances: its |
| + lifecycle field, the refusals and the recovery contract. Reuse its |
| + conclusions where they apply. Its refusal of fleets and pools does not |
| + carry over: rootfs apps live only on fleets and pools (R1). |
| +- The plans-caps never-sleep refusals (plans 3b, not built) must not |
| + refuse a rootfs app on its own boxes. A customer's own box costs |
| + Carloku nothing idle. |
| + |
| +### 5.7 Where a rootfs row may run (R1) |
| + |
| +- **Shared deployments** (carloku.com): only on a fleet box |
| + (`CARLOS_REGION=fleet/<name>`) or a box serving only non-shared pools |
| + (`CARLOS_POOLS` without `""`). |
| +- **Single-tenant deployments**: a deployment-level setting declares |
| + that every box belongs to one customer, and then any box qualifies. |
| + The builder names the setting; it is operator-only and lives with the |
| + deployment's other settings. |
| +- **Every deployment:** the box must be nested-ready and must pass the |
| + rootfs probe. A rootfs row on a flat or unready box refuses to start |
| + and says why. |
| +- Enforced in two places: the console refuses to create or place a |
| + rootfs app where it may not run, and the box refuses to start one. |
| + The box check is the one that counts; the console check is for a |
| + clear message. |
| +- The `carlos status --layout` warning about flat neighbours (nested |
| + isolation not in force) applies to a fleet box too. On a box owned by |
| + one customer the neighbours are that customer's own, so this is a |
| + warning, not a refusal. |
| + |
| +### 5.8 The local check (R3) |
| + |
| +`carlos ship --check <source>` (name is my placeholder) runs the image on |
| +the developer's machine the way a box would, using whatever local |
| +container runtime they have (Docker or Podman). It is a convenience, |
| +not a security boundary: |
| + |
| +- read-only root, a writable directory at the instance path, a private |
| + `/tmp`; |
| +- `CARLOS_SOCKET` and `CARLOS_DB` set, the other `CARLOS_*` vars set to |
| + plausible local values; |
| +- the default memory tier as the memory limit; |
| +- `GET /` on the socket within the health bound; |
| +- each worker process started and seen alive after a few seconds; |
| +- a clear report of what failed: binding a port instead of the socket, |
| + writing outside the instance directory, running out of memory, an |
| + amd64 image. |
| + |
| +`carlos ship` runs the same static checks (architecture, declared |
| +processes, image size) before uploading. Only `--check` needs a local |
| +runtime. |
| + |
| +### 5.9 Reaching the app's own services |
| + |
| +- **Public endpoints** (S3, a managed database with a public endpoint): |
| + reachable today through the egress fence. |
| +- **Private addresses** (an RDS or ElastiCache endpoint at `10.x` in the |
| + customer's VPC): **blocked today.** The nested `IPAddressDeny=` covers |
| + RFC 1918, and no per-app egress allow exists on main. Moving an app |
| + whose database is private needs one. This design asks for it as a |
| + capability that is valid only for rootfs and binary rows on the |
| + customer's own boxes (R1): a list of CIDRs or hostnames the operator |
| + of that deployment grants, rendered as `IPAddressAllow=` on the |
| + instance's slice. The box's own addresses, IMDS and the box's |
| + loopback stay denied whatever the list says. |
| +- **Uploads on Carloku**: `carlos store create`, no change. |
| +- **An existing bucket** the app already uses: keys through `carlos env |
| + set`. No change. |
| + |
| +## 6. Order of work |
| + |
| +1. **A Rails 8 sample app on a Carloku fleet box.** SQLite with Solid |
| + Queue, Cache and Cable in the primary database, uploads through |
| + `carlos store`, one `web` and one `worker` process. Proves §5.1 to |
| + §5.8. Neutral names in everything (`shop`, `example.com`). |
| +2. **A tenant's own single-tenant deployment goes nested.** Separate |
| + work, outside this design. It is what makes isolation real there. |
| +3. **The Rails app off Kubernetes** onto that deployment: MySQL and |
| + Redis in the same region through §5.9's egress allow, Sidekiq as a |
| + declared worker, existing uploads through env keys. Whether it also |
| + needs stateless replicas first is §13 Q1. |
| + |
| +### 6.1 What the sample app must show |
| + |
| +- `rails new` with SQLite, then Solid Queue, Cache and Cable configured |
| + to share the primary database at `CARLOS_DB`. |
| +- Puma bound to `unix://$CARLOS_SOCKET`. |
| +- A job enqueued by a request and run by the worker. |
| +- An upload stored in the app's `carlos store` bucket. |
| +- A deploy of a second version with the first one's data intact. |
| +- A crash of the worker and of `web`, each recovered without a visitor. |
| + |
| +## 7. Correctness rules |
| + |
| +- C1. A rootfs row never starts flat, never starts on a box that has not |
| + passed the rootfs probe, and never starts on a shared box (§5.7). |
| +- C2. An old box never stages a rootfs artifact as a binary (§5.1). |
| +- C3. A rootfs instance's root is its own image and nothing else's. No |
| + path inside the namespace reaches another app's image, build or data. |
| +- C4. The trampoline runs a rootfs command only from the staged manifest, |
| + never from tenant-writable config (§5.2). |
| +- C5. Every rootfs process (web, workers, seed, run) is inside the |
| + instance's slice and tier. Nothing the image starts escapes the cgroup. |
| +- C6. The egress allow (§5.9) can only add the listed destinations; it |
| + never reopens IMDS, the box's loopback or the box's own addresses. |
| +- C7. One writer still holds: one box runs a given instance, under the |
| + lease, always-on or not. |
| + |
| +## 8. Tests that prove it |
| + |
| +- The isolation probe in rootfs mode, run on a dev box, with each |
| + property it checks today. |
| +- Unit-file tests: the rootfs variants share the namespace block's |
| + restated properties with the probe, as |
| + `TestInstanceTemplatesShareOneNamespace` does today. |
| +- Adopt: a previous release's adopt pass against a rootfs-only manifest |
| + stages nothing (C2). |
| +- Placement refusals: shared box, flat box, unready box (C1), each from |
| + the box side with the console check bypassed. |
| +- Trampoline: a rootfs launch with a command not from the staged |
| + manifest refuses (C4). |
| +- Always-on: boot start, crash restart and adoption restart with no |
| + request in between; replication stays with the activator. |
| +- The sample app end to end on a dev fleet box (§6.1). Production-wired |
| + witnesses, not fixtures: the claims go through real wiring. |
| + |
| +## 9. Docs |
| + |
| +- A guide: "Run a Rails app on your own boxes", with the Dockerfile, |
| + `config/puma.rb`, the `database.yml` sharing one file, and the |
| + manifest's `processes`. |
| +- A short Laravel and a short Django variant once step 1 works. |
| +- The contract reference: `CARLOS_SOCKET`, `CARLOS_DB`, health, writable |
| + paths, memory tiers, no JIT. |
| +- All user-facing strings go through copy review before they are |
| + written. |
| + |
| +## 10. Provider boundary |
| + |
| +Nothing in the rootfs path is AWS-specific. The image, the unit |
| +variant, the process declaration and the local check are plain Linux |
| +and systemd. The egress allow takes CIDRs and hostnames, not AWS |
| +resource IDs. Managed databases are the customer's choice of endpoint. |
| + |
| +## 11. Calls I made (for review) |
| + |
| +- OCI image or directory as input, flattened locally. Buildpacks work |
| + because they produce OCI images. Docker is not required to ship, only |
| + to run `--check`. |
| +- `RootImage=` over `RootDirectory=`: one file to stage, checksum and |
| + delete, and read-only by construction. |
| +- Workers as co-spawn sidecars, not standalone members, because they |
| + must share the instance's database. |
| +- Always-on rows stay activator-owned and nested. |
| +- Upload cap of 2 GiB for the rootfs part; worker cap of 4 per instance. |
| +- Health bound of at most 120 s for rootfs apps. |
| +- Moves refuse rootfs rows in v1. |
| +- Seeds and `carlos run` for rootfs are desirable, not required for v1. |
| +- The egress allow is in this design because step 3 cannot happen |
| + without it, but it is not needed for step 1 and can land after it. |
| + |
| +## 12. Follow-ups (separate designs) |
| + |
| +- **Stateless replicas**: N copies of an app with no local state, on |
| + several boxes, behind the edge or a load balancer, with no lease and |
| + no litestream. Rootfs apps that declare no local database are the |
| + natural first users. Region HA's dormant load-balancer health answer |
| + and private forwarding are a starting point. |
| +- **JIT opt-in**: a capability that drops `MemoryDenyWriteExecute` for |
| + one app, on its own boxes only. |
| +- **More than one SQLite file per instance**. |
| +- **Rootfs on shared boxes**, once the variant has earned trust. |
| + |
| +## 13. Open questions for Paul |
| + |
| +- **Q1.** For the Rails app moving off Kubernetes (step 3), is failover |
| + within minutes acceptable at first, or must it run on several boxes |
| + from day one? If the latter, stateless replicas (§12) move ahead of |
| + step 3. |
amadan