carlos / platform Public

One tenant can fill /data and take the whole box down: needs a dedicated /data with per-instance project quotas open

Found by the 2026-10 tenant-isolation work (#214 item 1, branch tenant-iso-hardening, Fable-reviewed).

/data is a directory on the root ext4, and every writer runs as uid carlos, so no tenant has its own quota. The hardening branch refuses new starts and seeds below max(1 GiB, 10%) free and logs a warning every 30 s. It deliberately stops nothing, because space held by unlinked-but-open files can't be traced to a tenant, and a stop would bypass the activator's fence.

What's left: a tenant that is already awake can still fill /data to zero. The registry SQLite, the agent's state and litestream's shadow WAL then all hit ENOSPC:

  • nothing on the box can wake;
  • replication stalls for every tenant, so durability is exposed until space is freed;
  • other tenants' writes fail.

SQLite handles ENOSPC atomically, so nothing is corrupted, but recovery is manual. It's a box-wide availability denial of service from one tenant.

Fix (an infrastructure change, needs scoped approval): give /data its own volume mounted with prjquota, and assign each instance directory a project ID with a quota. Until then, surface the low-disk state on the status page or as an alert (#… alerts-sns landed on main) rather than only in the log, before hostile tenants are admitted.

Tracked under #247 (tenant isolation before Carloku takes outside tenants).