amadan

carlos / platform Public

edge: memory wall on the unit tree — MemoryHigh/Max + OOMPolicy=continue (memory-pressure spec §2.2)

Paul Campbell pushed by paul@keymail.dev dd6607632b26aa2e4486217f521e716326e1215e parent d6cd0e9
2 files changed, +28 −2
  • internal/host/host_test.go +8 −0
  • internal/host/units/carlos-edge.service +20 −2
diff --git a/internal/host/host_test.go b/internal/host/host_test.go
index 6f442c850..aba8c5a50 100644
--- a/internal/host/host_test.go
+++ b/internal/host/host_test.go
@@ -123,6 +123,14 @@ func TestBootstrapWritesUnitsToStagingDir(t *testing.T) {
// cgroup manager degrades to nil and every exec-backed tenant goes
// blind on the Instances page.
"Delegate=cpu memory io pids",
+ // The outer wall (memory-pressure spec §2.2): the edge tree is
+ // where every exec-backed tenant lives, so this is the one place a
+ // runaway tenant meets a ceiling before the kernel starts thrashing.
+ // OOMPolicy=continue is the line that keeps one tenant's OOM kill
+ // from becoming a whole-edge restart (systemd's default is stop).
+ "MemoryHigh=70%",
+ "MemoryMax=80%",
+ "OOMPolicy=continue",
} {
if !strings.Contains(string(edge), want) {
t.Errorf("carlos-edge.service: missing %q", want)
diff --git a/internal/host/units/carlos-edge.service b/internal/host/units/carlos-edge.service
index ee57b7f40..461c6a01e 100644
--- a/internal/host/units/carlos-edge.service
+++ b/internal/host/units/carlos-edge.service
@@ -54,8 +54,26 @@ MemoryDenyWriteExecute=yes
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
SystemCallArchitectures=native
-# No MemoryMax: hibernating instances are this unit's children, so a cap here
-# would cap the whole set of awake instances at once.
+# The outer wall (memory-pressure spec 2026-08-28 §2.2). Exec-backed
+# instances are this unit's children, so these DO cap the whole set of
+# awake instances at once — that is now the point: on 2026-08-28 the
+# flagship thrashed for 15 minutes with no swap, no ceiling and an OOM
+# killer that never fired, because nothing bounded the tree. Percentages
+# so one file fits a t4g.micro and a t4g.medium; 80% leaves what the
+# box's other units (keymail, console, ssm, litestream, journald) use.
+# Past High the kernel reclaims the tree (page cache first, then zram);
+# at Max it kills the largest process in the tree — a tenant child, by
+# two orders of magnitude over the agent. Per-instance caps below this
+# (carlos-exec@<host> memory.max, internal/cgroup) are the inner walls.
+MemoryHigh=70%
+MemoryMax=80%
+# systemd's default OOMPolicy=stop STOPS the whole service when the kernel
+# OOM-kills ANY process in the unit's cgroup tree (memory.events is
+# hierarchical, delegation or not). Restart=always would bring the edge
+# back a second later and cycle every awake instance on the box. With
+# continue, one tenant's kill is one tenant's kill; the activator recycles
+# it on its next request.
+OOMPolicy=continue
TasksMax=1024
# Exec-backed instances run as this unit's children. Delegation hands the
# unit its own cgroup subtree — chowned to User=carlos, unmanaged by systemd