carlos / platform Public

Isolation the nested layout does not give: CPU/IO/disk limits, shared network, readable binaries, private route addrs, trusted XFF Issue closed

On branch tenant-iso-hardening

Claimed by claude~paul@keymail.dev 3 weeks ago

Found by the 2026-10-03 tenant-isolation audit of main 78fc8d40, by reading code. These survive the nested layout (#126), so they need their own fixes.

  1. No CPU, IO or pids limits. Flat children get only memory.max/memory.high (internal/cgroup/cgroup.go:51-62, 307-312). Nested units get TasksMax=256 but no CPU or IO weight or cap (instancectl/server.go:243 sets memory only). One tenant can starve every other tenant on the box. /data has no per-tenant quota, so one tenant can fill the disk and break the registry, every database and litestream.
  2. Every tenant can read every app's binaries. /opt/carlos/versions and its contents are 0755 (internal/release/versions.go:139-150, host/bootstrap.go:429-430), and the instance templates do not hide /opt/carlos. Any tenant can read another app's binary, anything built into it, and the account and app names on the box.
  3. One network namespace for everyone. No template sets PrivateNetwork or an IPAddressDeny beyond IMDS. A tenant can reach loopback services directly, such as the console on 127.0.0.1:8090 on the console box, other tenants' TCP addr upstreams, and private addresses of other boxes. That bypasses the edge's gates and door keys. It can also take a loopback port that an addr route expects while the route's owner is down.
  4. A member can point their own route at internal addresses. addrs accepts loopback and RFC 1918 addresses (console/apiinstances.go:1356; provision.go:612-622 and validAddrsElement refuse only link-local). The edge then proxies public requests to services local to that box, or to other tenants' private upstreams. The comment there says this was "escalated separately"; I couldn't find where.
  5. Console trusts X-Forwarded-For from any loopback peer (console/authcli.go:890-913). A tenant on the console box can send a fresh IP per request and get past the per-IP limits on device-code login.

Fixes are going on a branch from this session.

Two branches cover this discussion. Items 1–3 are on tenant-iso-hardening, which this is now attached to. Items 4–5 are on tenant-iso-routes (012d74ef, Fable-approved, waiting on Paul's choice between members having no upstreams at all and members allowed any public IP; see #50).

Items 1–3 merged to main as 09814f7 (tenant-iso-hardening; Fable review plus one re-check, CI green). The disk residual is tracked in #245. Items 4–5 are still on tenant-iso-routes, waiting on Paul's upstream decision.

Items 4–5 merged to main as 1c43c2c (tenant-iso-routes; Fable review plus two re-checks, copy approved by Paul, CI green). All five items are now on main. The disk residual is tracked in #245.