carlos / platform Public

adopt never garbage-collects /opt/carlos/versions — tito IE box hit 97% disk and wedged Feature Request open

Migrated from github.com/carlosframework/platform issue #221, opened 2026-08-17. GitHub issues are closed for this repo; see AGENTS.md, "Work is tracked on amadan, not GitHub".

Still outstanding: adopt never garbage-collects /opt/carlos/versions; the tito box hit 97% disk and wedged.

---

What happened: the tito IE box (i-025a0b70f3d93304d, 40 GB root) accumulated ~900 release dirs in /opt/carlos/versions/ (~40 MB each, ~35 GB total — console + titogo + vitogo + keymail, every version ever shipped). Disk reached 97% / 1.4 GB free, and the box wedged twice under it: SSM document-worker crashes (ipc messaging received timeout signal) on 2026-08-16 21:17Z and 2026-08-17 17:31Z, the second escalating to a full userspace wedge — console, titogo edge (Tito production), and keymail all timing out while EC2 status checks stayed green. Recovered by reboot ~2026-08-17 18:00Z. Kernel log shows no OOM; memory was fine; CPU credits nearly full. The graveyard is the illness.

Ask: release GC as part of the adoption pass (and/or a carlos gc operator verb — today the CLI has neither gc nor prune). Safety rules it must respect:

  • never delete a version referenced by any live/<host> symlink
  • never delete a version any channel currently points at (rollback target)
  • keep a rollback window (e.g. last N days or last K versions per app)

Interim: a conservative manual sweep is being run on the tito box (delete only dirs unreferenced by live/* AND older than 14 days, dry-run first). That's a bandage; every long-lived CARLOS box will hit this.

🤖 Generated with Claude Code