amadan

carlos / platform Public

Scheduled ticks must not hold an instance awake (#371)

* spec: scheduled ticks must not hold an instance awake

A 5-minute schedule on a 15-minute idle window means the instance never
hibernates: the tick goes through ServeHTTP like a visitor, so every fire
stamps lastActive and buys another full window for a 1ms 204. Found on the
flagship, where jam.bac has been resident continuously since its schedule
landed.

Three parts: a non-stamping waking entry point for the deliverer, a guard
so the fix doesn't trade one resident process for 288 wake-path traversals
a day, and a holds_awake field so the member can see it.

Deliberately inert on the flagship as configured — the behaviour change
arrives with the separate 15m -> 90s idle-window branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NbP6qc2gf7BFZkQ8QLcm4C

* spec: record why the guard must not read routes.next_due

That column is written only at sleep and left stale on purpose while a host
is awake, so a guard reading it would compare against a past timestamp and
never sleep again. Looks like free coverage; is a wedge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NbP6qc2gf7BFZkQ8QLcm4C

* plan: scheduled ticks and hibernation, seven tasks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NbP6qc2gf7BFZkQ8QLcm4C

* plan: correct test scaffolding to the helpers that actually exist

Pre-flight scan against the real test files: openTestRegistry not
testRegistry, testActivator/testRoute/fakeClock/waitFor not invented
fixtures, wakeFixture in internal/host, renderScheduleInstance in the CLI.
Task 3 also picks up the one existing test the guard changes — the wake
story's frozen 2026-08-23 clock makes every row permanently overdue against
the activator's real one, which the guard would read as 'a tick is imminent,
stay awake'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NbP6qc2gf7BFZkQ8QLcm4C

* registry: EarliestTimers, one query for the hibernation guard

* activator: a scheduled tick wakes and holds, but earns no activity credit

* activator: don't hibernate into a tick due inside the idle window

* activator: guard falls through to convergence; fix imminent-tick test to cover a future tick

Fix round 1: the guard's continue was skipping the build-convergence
check, so a host held awake by a tight schedule would serve a stale
binary forever after a ship (Bounce only fires on config change).
Fall through instead — a tick-held host never reaches a next sleep,
so it is in the same always-awake class the convergence arm already
covers for sidecars. Add TestSweepHeldForTickStillConverges to pin
it.

Also fix TestSweepKeepsHostAwakeWhenATickIsImminent, which armed the
timer 5 minutes in the past relative to sweep time (an overdue tick,
already covered by TestSweepKeepsHostAwakeForAnOverdueTick) rather
than a genuinely future one inside the window. Arm 21 minutes out
before advancing the clock 16 minutes so the tick is 5 minutes in
the future at sweep time.

Update the spec and plan doc paragraphs that carried the overruled
'converges at its next genuine sleep' reasoning.

* host: schedule ticks deliver through the non-stamping activator door

* schedule: report when a schedule's interval holds its instance awake

* schedule: gate holds_awake on the route's own hibernate flag, not just the box's idle window

* cli: schedule ls says when a schedule holds its instance awake

* docs: correct four claims the branch outgrew

---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Paul Campbell pushed by paul@keymail.dev f10d8e1d51a211aba0f18632ea6ee37ce20fe321 parent 32e8662
16 files changed, +2040 −30
  • docs/superpowers/plans/2026-08-31-scheduled-ticks-and-hibernation.md +1029 −0
  • docs/superpowers/specs/2026-08-30-scheduled-ticks-and-hibernation-design.md +315 −0
  • internal/activator/activator.go +77 −11
  • internal/activator/activator_test.go +253 −0
  • internal/host/schedule_wake_test.go +74 −1
  • internal/host/schedule_wiring.go +25 −6
  • internal/registry/timers.go +27 −0
  • internal/registry/timers_test.go +39 −0
  • internal/schedule/control.go +11 −5
  • internal/schedule/converge.go +61 −6
  • internal/schedule/converge_test.go +56 −0
  • internal/schedule/runner.go +2 −1
  • internal/schedule/status.go +7 −0
  • internal/schedule/status_test.go +14 −0
  • schedule.go +7 −0
  • schedule_test.go +43 −0
diff --git a/docs/superpowers/plans/2026-08-31-scheduled-ticks-and-hibernation.md b/docs/superpowers/plans/2026-08-31-scheduled-ticks-and-hibernation.md
new file mode 100644
index 000000000..dd80b082c
--- /dev/null
+++ b/docs/superpowers/plans/2026-08-31-scheduled-ticks-and-hibernation.md
@@ -0,0 +1,1029 @@
+# Scheduled ticks and hibernation — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Stop a platform-delivered scheduled tick from resetting an instance's idle clock, without trading one resident process for hundreds of wake-path traversals a day.
+
+**Architecture:** Three independent pieces. (1) The activator gains a non-stamping sibling of `ServeHTTP` that the schedule deliverer calls instead, so a tick wakes and holds the instance but never credits it as activity. (2) `sweepOnce` gains a guard that refuses to hibernate a host whose next platform timer is due sooner than the idle window, fed by one new pre-lock registry query. (3) The box computes whether a schedule's interval keeps its instance resident and reports it, so `carlos schedule ls` can say so.
+
+**Tech Stack:** Go 1.x, SQLite (`internal/registry`), no new dependencies.
+
+**Spec:** `docs/superpowers/specs/2026-08-30-scheduled-ticks-and-hibernation-design.md`
+
+## Global Constraints
+
+- **This change must be inert on the flagship as configured.** Idle is 15m, the tightest schedule on the fleet is 5m, and the guard's answer at those numbers is "stay awake". Any test or manual check that shows `jam.bac.oncarlos.com` beginning to hibernate on this branch is a bug, not a win. (Spec §4.)
+- **Wire shapes are additive.** A `schedule-status/<host>.json` written before this change must still parse. Never remove or retype an existing field. (AGENTS.md, "Rules this repo has paid for".)
+- **Build gate before any commit lands:** `go build ./...`, `go vet ./...`, `go test ./... -count=1`, `gofmt -l .` clean over the whole tree. `make test` covers the last three.
+- **Run Go commands with `TMPDIR=/tmp`**, or three unrelated socket tests fail with phantom errors (they fail on untouched `main` too). Sandbox note from AGENTS.md: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off`.
+- **CI is the `amadan` remote, not GitHub.** A branch pushed only to `origin` has been gated by nothing. Push to `amadan` and read the run before calling it green.
+- **`MinEvery = time.Minute`** (`internal/schedule/spec.go:20`) — the tightest schedule anyone can declare. Worst case for any calculation here is 1,440 ticks a day.
+- **Nothing in this plan adds a registry column, a tuning constant, a wire marker, or a `NextDue` change.** All four were considered and rejected in the spec (§2, §3.1, §5).
+
+---
+
+### Task 1: `EarliestTimers` — one query the guard can afford
+
+The guard needs "when is the next platform tick due, per host" as a single read it can do *before* taking the activator's mutex. `sweepOnce` already establishes that pattern for `DueInstances` and says why: "it's a registry read, and nothing under a.mu should wait on SQLite" (`internal/activator/activator.go:911`).
+
+**Files:**
+- Modify: `internal/registry/timers.go` (append after `DueTimers`)
+- Test: `internal/registry/timers_test.go`
+
+**Interfaces:**
+- Consumes: nothing from earlier tasks.
+- Produces: `func (r *Registry) EarliestTimers() (map[string]int64, error)` — host → the smallest `next_at > 0` among that host's timer rows. Hosts with no armed rows are absent from the map. Task 3 consumes it.
+
+- [ ] **Step 1: Write the failing test**
+
+In `internal/registry/timers_test.go`, following the file's existing setup helper for a temp registry:
+
+```go
+func TestEarliestTimersPerHost(t *testing.T) {
+ r := openTestRegistry(t)
+ rows := []Timer{
+ // Two rows for one host: the earliest wins.
+ {Host: "a.example", Name: "late", Kind: "schedule", Spec: "every 1h", Path: "/x", NextAt: 5000, DueAt: 5000},
+ {Host: "a.example", Name: "soon", Kind: "schedule", Spec: "every 5m", Path: "/y", NextAt: 1200, DueAt: 1200},
+ // A second host is reported independently.
+ {Host: "b.example", Name: "only", Kind: "schedule", Spec: "every 5m", Path: "/z", NextAt: 3000, DueAt: 3000},
+ // next_at == 0 means "not armed" and must never appear.
+ {Host: "c.example", Name: "parked", Kind: "schedule", Spec: "every 5m", Path: "/p", NextAt: 0, DueAt: 0},
+ }
+ for _, tm := range rows {
+ if err := r.UpsertTimer(tm); err != nil {
+ t.Fatalf("UpsertTimer %s/%s: %v", tm.Host, tm.Name, err)
+ }
+ }
+
+ got, err := r.EarliestTimers()
+ if err != nil {
+ t.Fatalf("EarliestTimers: %v", err)
+ }
+ want := map[string]int64{"a.example": 1200, "b.example": 3000}
+ if !reflect.DeepEqual(got, want) {
+ t.Errorf("EarliestTimers = %v, want %v", got, want)
+ }
+}
+
+func TestEarliestTimersEmpty(t *testing.T) {
+ r := openTestRegistry(t)
+ got, err := r.EarliestTimers()
+ if err != nil {
+ t.Fatalf("EarliestTimers: %v", err)
+ }
+ if len(got) != 0 {
+ t.Errorf("EarliestTimers on an empty table = %v, want empty", got)
+ }
+}
+```
+
+`openTestRegistry(t) *Registry` is the package's existing helper (`registry_test.go:469`) — verified, use it. There is no `must` helper in this package; the explicit error check above is the house style. `reflect` must be in the test file's imports.
+
+- [ ] **Step 2: Run the test and watch it fail**
+
+Run: `TMPDIR=/tmp go test ./internal/registry -run TestEarliestTimers -count=1 -v`
+Expected: FAIL — `r.EarliestTimers undefined (type *Registry has no field or method EarliestTimers)`.
+
+- [ ] **Step 3: Implement it**
+
+Append to `internal/registry/timers.go`:
+
+```go
+// EarliestTimers is the hibernation guard's batch: for every host with at
+// least one armed row, the soonest next_at among them. One query, because
+// the activator's sweep reads it BEFORE taking its mutex (activator.go's
+// "nothing under a.mu should wait on SQLite") and a per-entry query inside
+// the sweep loop would put SQLite exactly where that comment forbids it.
+//
+// next_at = 0 is "not armed" — a parked cron, or a row converge wrote and
+// the runner has not yet dated — and is excluded here rather than by the
+// caller, matching DueTimers' own `next_at > 0` gate.
+func (r *Registry) EarliestTimers() (map[string]int64, error) {
+ rows, err := r.sql.Query(`SELECT host, MIN(next_at) FROM timers WHERE next_at > 0 GROUP BY host`)
+ if err != nil {
+ return nil, err
+ }
+ defer rows.Close()
+ out := make(map[string]int64)
+ for rows.Next() {
+ var host string
+ var next int64
+ if err := rows.Scan(&host, &next); err != nil {
+ return nil, err
+ }
+ out[host] = next
+ }
+ return out, rows.Err()
+}
+```
+
+- [ ] **Step 4: Run the test and watch it pass**
+
+Run: `TMPDIR=/tmp go test ./internal/registry -run TestEarliestTimers -count=1 -v`
+Expected: PASS, both tests.
+
+- [ ] **Step 5: Full package gate, then commit**
+
+```bash
+TMPDIR=/tmp go test ./internal/registry -count=1
+gofmt -l internal/registry
+git add internal/registry/timers.go internal/registry/timers_test.go
+git commit -m "registry: EarliestTimers, one query for the hibernation guard"
+```
+
+---
+
+### Task 2: A tick that wakes but does not count
+
+`activatorDeliverer.Deliver` reaches the instance through `Service.ServeHTTP` (`internal/host/schedule_wiring.go:330`), the same door the edge uses, so `beginRequest`'s release stamps `lastActive` (`internal/activator/activator.go:594`) and every tick buys another full idle window.
+
+The fix mirrors `BeginSidecarRequest` (`activator.go:526`) — activity-invisible traffic — and differs in one way: a tick *wakes* a sleeping instance where a sidecar request is refused.
+
+**Files:**
+- Modify: `internal/activator/activator.go` (`beginRequest`, `releaseFunc`, `ServeHTTP`)
+- Test: `internal/activator/activator_test.go`
+
+**Interfaces:**
+- Consumes: nothing from earlier tasks.
+- Produces:
+ - `func (a *Service) ServeScheduled(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool` — same contract as `ServeHTTP` (returns false for a route it does not manage) but the request earns no activity credit. Task 4 consumes it.
+ - Unexported: `func (a *Service) begin(ctx context.Context, rt registry.Route, stamp bool) (release func(), err error)` and `func (a *Service) serve(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler, stamp bool) bool`.
+
+- [ ] **Step 1: Write the failing test**
+
+In `internal/activator/activator_test.go`, using whatever harness the existing wake/sleep tests use to build a `*Service` with a fake backend and a controllable clock:
+
+```go
+// A scheduled tick must not move lastActive: it is the platform's own clock
+// poking the instance, not a visitor. A tick that counted would make any
+// schedule tighter than the idle window a permanent keep-awake.
+func TestServeScheduledDoesNotStampActivity(t *testing.T) {
+ const host = "tick.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, time.Hour)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt) // a real beginRequest round trip, then release
+
+ a.mu.Lock()
+ before := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ clk.advance(time.Minute)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "http://carlos/jobs/sweep", nil)
+ if !a.ServeScheduled(rec, req, rt, okHandler()) {
+ t.Fatal("ServeScheduled returned false for a managed route")
+ }
+
+ a.mu.Lock()
+ after := a.entries[rt.Host].lastActive
+ inflight := a.entries[rt.Host].inflight
+ a.mu.Unlock()
+
+ if !after.Equal(before) {
+ t.Errorf("lastActive moved %v -> %v; a scheduled tick must not count as activity", before, after)
+ }
+ if inflight != 0 {
+ t.Errorf("inflight = %d after delivery, want 0 (the hold is released even though the stamp is not written)", inflight)
+ }
+}
+
+// The other half of the same coin, asserted against the first so the two
+// paths can never quietly become one again — which is the bug this fixes.
+func TestServeHTTPStillStampsActivity(t *testing.T) {
+ const host = "visitor.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, time.Hour)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ a.mu.Lock()
+ before := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ clk.advance(time.Minute)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("GET", "http://carlos/", nil)
+ if !a.ServeHTTP(rec, req, rt, okHandler()) {
+ t.Fatal("ServeHTTP returned false for a managed route")
+ }
+
+ a.mu.Lock()
+ after := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ if !after.After(before) {
+ t.Errorf("lastActive did not move on a visitor request (%v -> %v)", before, after)
+ }
+}
+
+// A tick is a wake source and must stay one: an instance asleep when its
+// timer comes due has to be woken by the delivery itself.
+func TestServeScheduledWakesASleepingInstance(t *testing.T) {
+ const host = "asleep.bab.oncarlos.com"
+ a, _, _, _ := testActivator(t, time.Hour) // no entries yet: the host is asleep
+ rt := testRoute(host)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "http://carlos/jobs/sweep", nil)
+ if !a.ServeScheduled(rec, req, rt, okHandler()) {
+ t.Fatal("ServeScheduled returned false for a managed route")
+ }
+ if rec.Code != http.StatusOK {
+ t.Fatalf("status %d, want 200 — the tick must wake the instance, not be refused", rec.Code)
+ }
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("instance is not awake after a scheduled delivery")
+ }
+}
+```
+
+**The helpers above are placeholders. These are the real ones, verified in `internal/activator/activator_test.go` — use them:**
+
+- `testActivator(t *testing.T, idle time.Duration) (*Service, *fakeBackend, *fakeClock, *registry.Registry)` (line 350) — builds the Service over the fake backend with `a.now = clk.now` already wired.
+- `testRoute(host string) registry.Route` (line 363) — a hibernate-flagged instance route.
+- `clk.advance(d)` (line 339) — moves the fake clock.
+- `waitFor(t, what string, cond func() bool)` (line 387) — poll for an async stop; `beginStopLocked` runs on its own goroutine, so never assert hibernation synchronously.
+
+To arrange an awake entry, do what `newTestServiceAwake` (line 375) does: a real `beginRequest` round trip, then `release()`. For the proxy argument, pass an inline `http.HandlerFunc` that writes the status you want to assert — do not dial `testRoute`'s placeholder socket.
+
+- [ ] **Step 2: Run the tests and watch them fail**
+
+Run: `TMPDIR=/tmp go test ./internal/activator -run 'TestServeScheduled|TestServeHTTPStillStamps' -count=1 -v`
+Expected: FAIL — `a.ServeScheduled undefined`.
+
+- [ ] **Step 3: Thread `stamp` through the three functions**
+
+In `internal/activator/activator.go`, rename `beginRequest` to `begin` and give it the flag. Keep a `beginRequest` wrapper so no other call site changes:
+
+```go
+// begin registers a request against rt's instance, waking it if asleep.
+//
+// stamp is what separates a visitor from the platform's own clock. Both take
+// the in-flight count — a delivery in progress must never be cut off by a
+// concurrent sweep — but only a visitor's release moves lastActive. A
+// scheduled tick that stamped would reset the idle window on every fire, so
+// any schedule tighter than that window would pin its instance awake forever
+// for however little work the handler does.
+func (a *Service) begin(ctx context.Context, rt registry.Route, stamp bool) (release func(), err error) {
+ // ... body unchanged from beginRequest, except the actAwake arm:
+ // return a.releaseFunc(e, stamp), nil
+}
+
+// beginRequest is begin for member traffic: activity counts.
+func (a *Service) beginRequest(ctx context.Context, rt registry.Route) (release func(), err error) {
+ return a.begin(ctx, rt, true)
+}
+```
+
+`releaseFunc` takes the flag and guards only the stamp — the decrement is unconditional:
+
+```go
+func (a *Service) releaseFunc(e *actEntry, stamp bool) func() {
+ var once sync.Once
+ return func() {
+ once.Do(func() {
+ a.mu.Lock()
+ e.inflight--
+ if stamp {
+ e.lastActive = a.now()
+ }
+ a.mu.Unlock()
+ })
+ }
+}
+```
+
+Then split `ServeHTTP`'s body into `serve` and give it two doors. Move the entire existing `ServeHTTP` body into `serve`, changing exactly one line — `a.beginRequest(r.Context(), rt)` becomes `a.begin(r.Context(), rt, stamp)`:
+
+```go
+func (a *Service) ServeHTTP(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
+ return a.serve(w, r, rt, proxy, true)
+}
+
+// ServeScheduled is ServeHTTP for a platform-delivered scheduled tick: same
+// wake, same in-flight hold, same error mapping — no activity credit.
+//
+// A dedicated method rather than a wire marker like SidecarHeader, and the
+// difference is the delivery path, not taste. The sidecar dials its
+// instance's public URL, so its traffic arrives through the edge and the
+// request is the only place a signal could ride. A tick never leaves the
+// process (schedule_wiring.go: "the tick never leaves the box"), so the call
+// site IS the signal, and there is no header for an outsider to send.
+func (a *Service) ServeScheduled(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
+ return a.serve(w, r, rt, proxy, false)
+}
+
+func (a *Service) serve(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler, stamp bool) bool {
+ // ... today's ServeHTTP body verbatim, with the one call swapped.
+}
+```
+
+Leave the sidecar branch inside `serve` exactly as it is. A tick cannot carry `SidecarHeader` (nothing sets it on the delivery request), so the branch is unreachable from `ServeScheduled` and costs nothing to keep in one place.
+
+- [ ] **Step 4: Run the tests and watch them pass**
+
+Run: `TMPDIR=/tmp go test ./internal/activator -run 'TestServeScheduled|TestServeHTTPStillStamps' -count=1 -v`
+Expected: PASS, all three.
+
+- [ ] **Step 5: Full package gate, then commit**
+
+The whole activator suite matters here — `begin` and `releaseFunc` are on every request path.
+
+```bash
+TMPDIR=/tmp go test ./internal/activator -count=1
+gofmt -l internal/activator
+git add internal/activator/activator.go internal/activator/activator_test.go
+git commit -m "activator: a scheduled tick wakes and holds, but earns no activity credit"
+```
+
+---
+
+### Task 3: The guard — don't sleep into an imminent tick
+
+Task 2 alone would turn a 5-minute schedule into 288 wake/sleep cycles a day, each a full SQLite checkpoint-and-restore traversal, on the path where #200 records an unclosed integrity gap. The guard is what makes Task 2 safe.
+
+**Threshold is `cfg.Idle` itself** — no new constant (spec §3.2, Paul's ruling). If the gap to the next tick is shorter than the window we would have stayed awake for anyway, staying awake *is* the cheaper answer.
+
+**Files:**
+- Modify: `internal/activator/activator.go` (`DueStore` interface, `sweepOnce`)
+- Test: `internal/activator/activator_test.go`
+
+**Interfaces:**
+- Consumes: `(*registry.Registry).EarliestTimers() (map[string]int64, error)` from Task 1.
+- Produces: `DueStore` gains `EarliestTimers() (map[string]int64, error)`. Any test fake implementing `DueStore` must grow the method.
+
+- [ ] **Step 1: Write the failing test**
+
+```go
+// The guard: a host whose next platform tick is nearer than the idle window
+// stays resident. Sleeping would buy a few minutes of absence at the price of
+// a full checkpoint-and-restore round trip, and then an immediate wake.
+func TestSweepKeepsHostAwakeWhenATickIsImminent(t *testing.T) {
+ const host = "imminent.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // Next tick in 5 minutes: inside the window.
+ armTimer(t, reg, host, clk.now().Add(5*time.Minute))
+ clk.advance(16 * time.Minute) // lastActive is now well past the window
+
+ a.sweepOnce()
+
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("host hibernated with a tick due inside the idle window")
+ }
+}
+
+// Beyond the window, the sleep is worth taking.
+func TestSweepHibernatesWhenTheNextTickIsFarOff(t *testing.T) {
+ const host = "faroff.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // Next tick in an hour: well beyond the window.
+ armTimer(t, reg, host, clk.now().Add(time.Hour))
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ waitFor(t, "the host to hibernate with its next tick an hour away", func() bool {
+ a.mu.Lock()
+ defer a.mu.Unlock()
+ return a.entries[rt.Host] == nil
+ })
+}
+
+// A host with no timer rows is untouched by the guard — the overwhelming
+// majority of instances, and the case that must not regress.
+func TestSweepHibernatesAHostWithNoTimers(t *testing.T) {
+ const host = "untimed.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, 15*time.Minute) // no timer rows at all
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ waitFor(t, "a host with no timers to hibernate as it always did", func() bool {
+ a.mu.Lock()
+ defer a.mu.Unlock()
+ return a.entries[rt.Host] == nil
+ })
+}
+
+// An overdue row (next_at already in the past — the runner has not got to it
+// yet) is the most imminent tick there is, so it holds the host too.
+func TestSweepKeepsHostAwakeForAnOverdueTick(t *testing.T) {
+ const host = "overdue.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // next_at already behind us: the runner will fire it within 15s.
+ armTimer(t, reg, host, clk.now().Add(-time.Minute))
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("host hibernated with an overdue tick pending")
+ }
+}
+```
+
+Write these two helpers once in `activator_test.go` and use them from all four tests (and from Task 2's, for `arrangeAwake`):
+
+```go
+// arrangeAwake drives rt's host to actAwake the way the rest of this file
+// does — a real beginRequest round trip — so lastActive is stamped and the
+// entry exists.
+func arrangeAwake(t *testing.T, a *Service, rt registry.Route) {
+ t.Helper()
+ release, err := a.beginRequest(context.Background(), rt)
+ if err != nil {
+ t.Fatalf("beginRequest %s: %v", rt.Host, err)
+ }
+ release()
+}
+
+// armTimer puts one armed schedule row on host, due at next.
+func armTimer(t *testing.T, reg *registry.Registry, host string, next time.Time) {
+ t.Helper()
+ if err := reg.UpsertTimer(registry.Timer{
+ Host: host, Name: "sweep", Kind: "schedule", Spec: "every 5m",
+ Path: "/jobs/sweep", NextAt: next.Unix(), DueAt: next.Unix(),
+ }); err != nil {
+ t.Fatalf("UpsertTimer %s: %v", host, err)
+ }
+}
+```
+
+`okHandler()` is the same idea for the proxy argument — an `http.Handler` writing 200 — and belongs beside them.
+
+**There is no `DueStore` fake in this package, and you do not need one.** `testActivator` (line 350) already wires a real `*registry.Registry` as the activator's `DueStore` and returns it. Insert real rows with `reg.UpsertTimer(registry.Timer{...})` and let the guard read them through the `EarliestTimers` query Task 1 added — that makes this test a genuine integration of the two halves rather than a mock of one. Arrange the awake entry as in Task 2, use `clk.advance` to push past the idle window, and assert hibernation with `waitFor` (the stop path is asynchronous).
+
+- [ ] **Step 2: Run the tests and watch them fail**
+
+Run: `TMPDIR=/tmp go test ./internal/activator -run TestSweep -count=1 -v`
+Expected: FAIL — the fake does not satisfy `DueStore`, or the guard tests hibernate when they should not.
+
+- [ ] **Step 3: Extend the interface and the sweep**
+
+In `internal/activator/activator.go`, add to `DueStore` (around line 173):
+
+```go
+type DueStore interface {
+ SetNextDue(host string, due int64) error
+ DueInstances(now int64) ([]registry.Route, error)
+ // EarliestTimers is the hibernation guard's input: host -> soonest armed
+ // timer. Read with DueInstances, before the sweep takes a.mu.
+ EarliestTimers() (map[string]int64, error)
+}
+```
+
+In `sweepOnce`, join the existing pre-lock read (immediately after the `DueInstances` call, still above `a.mu.Lock()`):
+
+```go
+ // The guard's input, read here for the same reason the due query is:
+ // nothing under a.mu may wait on SQLite. A failure degrades to "no
+ // timers", i.e. today's behaviour — a guard that cannot read must not
+ // pin every instance on the box awake.
+ earliest, err := a.due.EarliestTimers()
+ if err != nil {
+ log.Printf("activator: earliest-timer sweep: %v — hibernating without the schedule guard", err)
+ earliest = nil
+ }
+```
+
+Then in the hibernate arm, replacing the existing `if e.inflight == 0 && ...` block:
+
+```go
+ if e.inflight == 0 && now.Sub(e.lastActive) >= a.cfg.Idle {
+ // Don't sleep into an imminent tick. A wake/sleep pair is a full
+ // checkpoint-and-restore round trip; spending one to be absent
+ // for less time than we would have idled anyway is a straight
+ // loss. The threshold is the idle window itself rather than a
+ // constant of its own: it is already the deployment's stated
+ // opinion about how long residency is worth holding.
+ if next, ok := earliest[e.host]; ok && time.Unix(next, 0).Sub(now) <= a.cfg.Idle {
+ continue
+ }
+ log.Printf("activator: %s idle %s — hibernating", e.host, now.Sub(e.lastActive).Truncate(time.Second))
+ a.beginStopLocked(e)
+ continue
+ }
+```
+
+**(overruled — Paul, review round 1, 2026-08-31)** The snippet above and the
+following paragraph originally had the guard's `continue` skip the
+convergence check below it, on the theory that a host held awake for a tick
+converges at its "next genuine sleep." That reasoning is wrong: a tick-held
+host, by construction, never reaches a next sleep — that is precisely what
+the guard guarantees — so it would serve a stale build forever after a
+`carlos ship`, with nothing else to close the gap (`Bounce` only fires on
+config change). The fix is to let the guard's decision fall through to the
+convergence check instead of skipping it:
+
+```go
+ if e.inflight == 0 && now.Sub(e.lastActive) >= a.cfg.Idle {
+ next, ok := earliest[e.host]
+ heldForTick := ok && time.Unix(next, 0).Sub(now) <= a.cfg.Idle
+ if !heldForTick {
+ log.Printf("activator: %s idle %s — hibernating", e.host, now.Sub(e.lastActive).Truncate(time.Second))
+ a.beginStopLocked(e)
+ continue
+ }
+ // Fall through to the convergence check below: a host held awake
+ // for its schedule never reaches a "next wake" to converge on, so
+ // it is now one of the always-awake hosts that arm was written
+ // for — the same class as a sidecar host that never sleeps.
+ }
+```
+
+A tick-held host is, in other words, exactly the class the convergence arm's
+own comment already names: "the always-awake ones — notably every sidecar
+host, which never sleeps." It now reaches that arm on every sweep, the same
+as a sidecar, so a ship behind a tick-held host converges within one grace
+window rather than never.
+
+- [ ] **Step 4: Amend the one existing test the guard changes**
+
+`internal/host/schedule_wake_test.go`'s `TestScheduledTickWakesASleepingInstanceAndItHibernatesAgain` freezes the Runner's clock at `time.Date(2026, 8, 23, 13, 7, 0, 0, time.UTC)` while the activator in `wakeFixture` runs on real wall time. After the tick fires, `MarkTimerFired` advances the row to the next 6h occurrence *after that frozen clock* — a timestamp days in the past relative to the activator. The new guard reads that as "a tick is overdue, stay awake", and the test's step 6 ("the sweep hibernates it") would spin to its 30s deadline and fail.
+
+The guard is right and the fixture is not: in production an overdue row is the most imminent tick there is, because the runner fires it within 15s. So fix the clock, not the guard. Change the one line:
+
+```go
+ now := time.Now().UTC()
+```
+
+Every other assertion still holds — `occurrence` stays `now.Unix() - 5`, so the `X-Carlos-Schedule-At` check is unaffected, and step 5's `tm.NextAt > now.Unix()` still holds because the next 6h boundary after a real now is still ahead of it.
+
+Do not change anything else in that file; Task 4 owns it next.
+
+- [ ] **Step 5: Run the tests and watch them pass**
+
+Run: `TMPDIR=/tmp go test ./internal/activator -run TestSweep -count=1 -v`
+Expected: PASS, all four.
+Run: `TMPDIR=/tmp go test ./internal/host -run TestScheduledTickWakes -count=1 -v`
+Expected: PASS — the amended test still proves the whole wake story.
+
+- [ ] **Step 6: Full package gate, then commit**
+
+```bash
+TMPDIR=/tmp go test ./internal/activator ./internal/host -count=1
+gofmt -l internal/activator internal/host
+git add internal/activator/activator.go internal/activator/activator_test.go internal/host/schedule_wake_test.go
+git commit -m "activator: don't hibernate into a tick due inside the idle window"
+```
+
+---
+
+### Task 4: Point the deliverer at the new door
+
+**Files:**
+- Modify: `internal/host/schedule_wiring.go:326-333` (the handler inside `Deliver`)
+- Test: `internal/host/schedule_wake_test.go`
+
+**Interfaces:**
+- Consumes: `(*activator.Service).ServeScheduled` from Task 2.
+- Produces: nothing new.
+
+- [ ] **Step 1: Write the failing test**
+
+In `internal/host/schedule_wake_test.go`. `lastActive` is unexported and lives in another package, so this test asserts the *behaviour* that flows from it rather than the field — which is the better test anyway, because it is the bug stated as an observation:
+
+```go
+// The bug, as a test: a schedule that fires more often than the idle window
+// used to pin its instance awake forever, because each tick stamped activity
+// and bought another full window. With the tick activity-invisible, the
+// instance sleeps between ticks and wakes again for the next one.
+//
+// No timer rows are armed here, deliberately: this exercises the delivery
+// leg alone. The sweep guard (which reads the timers table) would correctly
+// hold a host awake for a tick due this soon, and that is Task 3's test.
+func TestDeliveryDoesNotHoldTheInstanceAwake(t *testing.T) {
+ const host = "ticky.bab.oncarlos.com"
+ const idle = 300 * time.Millisecond
+ reg, inst, svc, _ := wakeFixture(t, host, idle, func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(http.StatusNoContent)
+ })
+ rt, err := reg.Get(host)
+ if err != nil {
+ t.Fatalf("get route: %v", err)
+ }
+ d := &activatorDeliverer{svc: svc}
+
+ // Tick twice per idle window for a few windows. Under the old accounting
+ // the instance could never reach the sweep's threshold; now it does,
+ // between ticks, and the next tick wakes it again.
+ deadline := time.Now().Add(20 * time.Second)
+ for inst.startCount() < 2 && time.Now().Before(deadline) {
+ req := httptest.NewRequest(http.MethodPost, "http://carlos/jobs/sweep", nil)
+ if _, err := d.Deliver(context.Background(), *rt, req); err != nil {
+ t.Fatalf("Deliver: %v", err)
+ }
+ time.Sleep(idle / 2)
+ }
+ if n := inst.startCount(); n < 2 {
+ t.Fatalf("the instance started %d time(s): it never hibernated between ticks, so the tick is still buying residency", n)
+ }
+}
+
+// The app-facing half of the tick contract is untouched by any of this.
+func TestDeliveryStillCarriesScheduleAtHeader(t *testing.T) {
+ const host = "hdr.bab.oncarlos.com"
+ seen := make(chan string, 1)
+ reg, _, svc, _ := wakeFixture(t, host, time.Hour, func(w http.ResponseWriter, r *http.Request) {
+ select {
+ case seen <- r.Header.Get("X-Carlos-Schedule-At"):
+ default:
+ }
+ w.WriteHeader(http.StatusNoContent)
+ })
+ rt, err := reg.Get(host)
+ if err != nil {
+ t.Fatalf("get route: %v", err)
+ }
+ d := &activatorDeliverer{svc: svc}
+
+ req := httptest.NewRequest(http.MethodPost, "http://carlos/jobs/sweep", nil)
+ req.Header.Set("X-Carlos-Schedule-At", "1788126300")
+ status, err := d.Deliver(context.Background(), *rt, req)
+ if err != nil {
+ t.Fatalf("Deliver: %v", err)
+ }
+ if status != http.StatusNoContent {
+ t.Fatalf("status = %d, want 204", status)
+ }
+ select {
+ case got := <-seen:
+ if got != "1788126300" {
+ t.Errorf("handler saw X-Carlos-Schedule-At = %q, want it passed through unchanged", got)
+ }
+ case <-time.After(10 * time.Second):
+ t.Fatal("the handler never ran")
+ }
+}
+```
+
+`wakeFixture(t, host, idle, handler) (*registry.Registry, *fakeInstance, *activator.Service, string)` (line 157) and `fakeInstance.startCount()` (line 108) are this file's existing helpers — verified, use them. `reg.Get(host)` returns `(*registry.Route, error)` (`internal/registry/registry.go:667`).
+
+- [ ] **Step 2: Run the tests and watch them fail**
+
+Run: `TMPDIR=/tmp go test ./internal/host -run 'TestDeliverer|TestDeliveryStill' -count=1 -v`
+Expected: FAIL on the first — activity is stamped, because `Deliver` still calls `ServeHTTP`.
+
+- [ ] **Step 3: Swap the call**
+
+In `internal/host/schedule_wiring.go`, inside `Deliver`'s handler:
+
+```go
+ h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ // ServeScheduled, not ServeHTTP: a tick wakes the instance and is
+ // held in flight for its duration, but must not reset the idle
+ // clock. Reports false for a route it does not manage (not an
+ // instance, or not hibernate-flagged); that route is proxied
+ // straight, the same call shape the edge itself uses.
+ if d.svc == nil || !d.svc.ServeScheduled(w, r, rt, proxy) {
+ proxy.ServeHTTP(w, r)
+ }
+ })
+```
+
+Then update the `activatorDeliverer` doc comment (`schedule_wiring.go:213-220`), whose first bullet now understates what happens — it should read that a hibernating instance wakes for its tick and is held awake for the tick's duration, **and that the tick alone does not keep it awake afterwards.**
+
+- [ ] **Step 4: Run the tests and watch them pass**
+
+Run: `TMPDIR=/tmp go test ./internal/host -run 'TestDeliverer|TestDeliveryStill' -count=1 -v`
+Expected: PASS, both.
+
+- [ ] **Step 5: Full package gate, then commit**
+
+```bash
+TMPDIR=/tmp go test ./internal/host -count=1
+gofmt -l internal/host
+git add internal/host/schedule_wiring.go internal/host/schedule_wake_test.go
+git commit -m "host: schedule ticks deliver through the non-stamping activator door"
+```
+
+---
+
+### Task 5: `holds_awake` — let the box say what it knows
+
+The box is the only place that holds both numbers: the timer rows and the activator's idle window. So the box decides and reports; the CLI only renders.
+
+**Files:**
+- Modify: `internal/schedule/status.go:24` (add the field)
+- Modify: `internal/schedule/converge.go:313-334` (`StatusFor`, `toStatus`, plus a new `holdsAwake`)
+- Modify: `internal/schedule/converge.go` (`Converger` struct: an `Idle` field) and `internal/schedule/runner.go:53` (`Runner` struct: an `Idle` field)
+- Modify: `internal/host/schedule_wiring.go` (`newScheduleDeps`: pass the window through)
+- Modify: `internal/activator/activator.go` (expose the window)
+- Test: `internal/schedule/converge_test.go`, `internal/schedule/status_test.go`
+
+**Interfaces:**
+- Consumes: nothing from earlier tasks.
+- Produces:
+ - `TimerStatus.HoldsAwake bool` with JSON tag `holds_awake`. Task 6 renders it.
+ - `func (a *Service) IdleWindow() time.Duration`.
+ - `StatusFor(reg *registry.Registry, rt registry.Route, box string, now time.Time, idle time.Duration) (Status, error)` — one added parameter.
+
+- [ ] **Step 1: Write the failing test**
+
+```go
+func TestHoldsAwakeWhenTheIntervalIsInsideTheIdleWindow(t *testing.T) {
+ now := time.Unix(1788120000, 0).UTC()
+ cases := []struct {
+ name string
+ kind string
+ spec string
+ idle time.Duration
+ want bool
+ }{
+ {"tighter than the window", KindSchedule, "every 5m", 15 * time.Minute, true},
+ {"exactly the window", KindSchedule, "every 15m", 15 * time.Minute, true},
+ {"wider than the window", KindSchedule, "every 1h", 15 * time.Minute, false},
+ {"tight, short window", KindSchedule, "every 5m", 90 * time.Second, false},
+ {"a once-timer never holds", KindOnce, "at 1788130000", 15 * time.Minute, false},
+ {"hibernation not armed", KindSchedule, "every 1m", 0, false},
+ {"hourly cron", KindSchedule, "cron 0 * * * *", 15 * time.Minute, false},
+ {"every-minute cron", KindSchedule, "cron * * * * *", 15 * time.Minute, true},
+ {"unparseable spec is not a claim", KindSchedule, "banana", 15 * time.Minute, false},
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ got := holdsAwake(registry.Timer{Kind: tc.kind, Spec: tc.spec}, tc.idle, now)
+ if got != tc.want {
+ t.Errorf("holdsAwake(%q, idle=%s) = %v, want %v", tc.spec, tc.idle, got, tc.want)
+ }
+ })
+ }
+}
+
+// The wire shape is additive: an object written before this field existed
+// must still parse, and read back as false.
+func TestStatusWithoutHoldsAwakeStillParses(t *testing.T) {
+ raw := []byte(`{"schema":1,"host":"h","box":"b","reported_at":"2026-08-30T21:40:05Z","timers":[{"name":"sweep","kind":"schedule","spec":"every 5m","path":"/jobs/sweep","next_at":1788126300}]}`)
+ var st Status
+ if err := json.Unmarshal(raw, &st); err != nil {
+ t.Fatalf("unmarshal a pre-holds_awake status: %v", err)
+ }
+ if len(st.Timers) != 1 || st.Timers[0].HoldsAwake {
+ t.Errorf("HoldsAwake = %v, want false for an object written before the field existed", st.Timers[0].HoldsAwake)
+ }
+}
+```
+
+- [ ] **Step 2: Run the tests and watch them fail**
+
+Run: `TMPDIR=/tmp go test ./internal/schedule -run 'TestHoldsAwake|TestStatusWithout' -count=1 -v`
+Expected: FAIL — `undefined: holdsAwake`, and `st.Timers[0].HoldsAwake` undefined.
+
+- [ ] **Step 3: Implement the field, the predicate and the plumbing**
+
+`internal/schedule/status.go`, appended to `TimerStatus` (additive, last field):
+
+```go
+ // HoldsAwake is the box's warning: this row fires at least as often as
+ // the deployment's idle window, so its instance never hibernates between
+ // ticks and accrues instance-hours around the clock. Computed by the box,
+ // which is the only place that holds both numbers. False from a box that
+ // predates the field, and false wherever hibernation is not armed.
+ HoldsAwake bool `json:"holds_awake,omitempty"`
+```
+
+`internal/schedule/converge.go`, next to `toStatus`:
+
+```go
+// holdsAwake reports whether t's interval is tight enough that its instance
+// never gets to sleep: the gap between consecutive occurrences is no wider
+// than idle, so the activator's guard (and, before it, the tick's own wake)
+// keeps the process resident indefinitely.
+//
+// The gap comes from two Next calls rather than Spec.Every, so a cron gets a
+// real answer instead of a special case. idle <= 0 means hibernation is not
+// armed on this box: nothing holds anything awake, because nothing sleeps.
+// A once-timer fires once and cannot hold anything. An unparseable spec is
+// not evidence of a problem, so it makes no claim.
+func holdsAwake(t registry.Timer, idle time.Duration, now time.Time) bool {
+ if idle <= 0 || t.Kind != KindSchedule {
+ return false
+ }
+ sp, err := ParseSpec(t.Spec)
+ if err != nil {
+ return false
+ }
+ first := sp.Next(now)
+ if first.IsZero() {
+ return false
+ }
+ second := sp.Next(first)
+ if second.IsZero() {
+ return false
+ }
+ return second.Sub(first) <= idle
+}
+```
+
+`toStatus` and `StatusFor` take the window:
+
+```go
+func StatusFor(reg *registry.Registry, rt registry.Route, box string, now time.Time, idle time.Duration) (Status, error) {
+ // ... unchanged, except:
+ // st.Timers = append(st.Timers, toStatus(r, idle, now))
+}
+
+func toStatus(t registry.Timer, idle time.Duration, now time.Time) TimerStatus {
+ return TimerStatus{
+ Name: t.Name, Kind: t.Kind, Spec: t.Spec, Path: t.Path,
+ NextAt: t.NextAt, DueAt: t.DueAt, LastAt: t.LastAt,
+ LastStatus: t.LastStatus, LastMs: t.LastMs, Attempts: t.Attempts,
+ HoldsAwake: holdsAwake(t, idle, now),
+ }
+}
+```
+
+Add `Idle time.Duration` to both `Converger` (`converge.go:36` area) and `Runner` (`runner.go:53` area), and pass it at each `StatusFor` call site (`converge.go:174`, `runner.go:179`).
+
+Expose the window from the activator, in `internal/activator/activator.go`:
+
+```go
+// IdleWindow is cfg.Idle, for the box-side reporters that must tell a member
+// whether their schedule is tight enough to hold the instance resident. Read
+// only; the window is fixed at construction from CARLOS_HIBERNATE_IDLE.
+func (a *Service) IdleWindow() time.Duration { return a.cfg.Idle }
+```
+
+And thread it in `newScheduleDeps` (`internal/host/schedule_wiring.go:66`), where `svc` may legitimately be nil:
+
+```go
+ // Zero when no activator is armed on this box: nothing hibernates, so no
+ // schedule can hold anything awake, and holdsAwake reports false for all
+ // of them.
+ var idle time.Duration
+ if svc != nil {
+ idle = svc.IdleWindow()
+ }
+```
+
+Set `Idle: idle` on both the `Runner` literal and the `NewConverger` call. If `NewConverger`'s signature has no room, add the field after construction — follow whichever shape that constructor already uses.
+
+- [ ] **Step 4: Run the tests and watch them pass**
+
+Run: `TMPDIR=/tmp go test ./internal/schedule -run 'TestHoldsAwake|TestStatusWithout' -count=1 -v`
+Expected: PASS, all cases.
+
+- [ ] **Step 5: Full gate across the three touched packages, then commit**
+
+```bash
+TMPDIR=/tmp go test ./internal/schedule ./internal/host ./internal/activator -count=1
+gofmt -l internal/schedule internal/host internal/activator
+git add internal/schedule internal/host/schedule_wiring.go internal/activator/activator.go
+git commit -m "schedule: report when a schedule's interval holds its instance awake"
+```
+
+---
+
+### Task 6: Say it in `carlos schedule ls`
+
+**Files:**
+- Modify: `schedule.go:337-350` (the per-timer render loop)
+- Test: `schedule_test.go`
+
+**Interfaces:**
+- Consumes: `schedule.TimerStatus.HoldsAwake` from Task 5.
+- Produces: nothing.
+
+- [ ] **Step 1: Write the failing test**
+
+In `schedule_test.go`, following how the existing render tests capture output:
+
+```go
+func TestScheduleLsFlagsAHoldsAwakeRow(t *testing.T) {
+ var b bytes.Buffer
+ now := time.Unix(1788126000, 0).UTC()
+ inst := schedule.Status{
+ Schema: 1, Host: "jam.bac.oncarlos.com", Box: "annie-welding",
+ ReportedAt: now.Format(time.RFC3339),
+ Timers: []schedule.TimerStatus{{
+ Name: "sweep", Kind: schedule.KindSchedule, Spec: "every 5m",
+ Path: "/jobs/sweep", NextAt: now.Add(2 * time.Minute).Unix(),
+ LastAt: now.Add(-3 * time.Minute).Unix(), LastStatus: "204", LastMs: 1,
+ HoldsAwake: true,
+ }},
+ }
+ renderScheduleInstance(&b, inst, now)
+
+ out := b.String()
+ if !strings.Contains(out, "sweep") {
+ t.Fatalf("row missing from output:\n%s", out)
+ }
+ if !strings.Contains(out, "never hibernates") {
+ t.Errorf("no holds-awake warning in output:\n%s", out)
+ }
+}
+
+func TestScheduleLsSaysNothingForAnOrdinaryRow(t *testing.T) {
+ var b bytes.Buffer
+ now := time.Unix(1788126000, 0).UTC()
+ inst := schedule.Status{
+ Schema: 1, Host: "seapointish.bag.oncarlos.com", Box: "annie-welding",
+ ReportedAt: now.Format(time.RFC3339),
+ Timers: []schedule.TimerStatus{{
+ Name: "sync", Kind: schedule.KindSchedule, Spec: "every 6h",
+ Path: "/jobs/sync", NextAt: now.Add(time.Hour).Unix(),
+ }},
+ }
+ renderScheduleInstance(&b, inst, now)
+
+ if strings.Contains(b.String(), "never hibernates") {
+ t.Errorf("warned about a 6-hourly schedule:\n%s", b.String())
+ }
+}
+```
+
+- [ ] **Step 2: Run the tests and watch them fail**
+
+Run: `TMPDIR=/tmp go test . -run TestScheduleLs -count=1 -v`
+Expected: FAIL — the warning string is absent.
+
+- [ ] **Step 3: Render it**
+
+In `schedule.go`, inside the per-timer loop, after the existing `fmt.Fprintf(w, " %-*snext %s last %s\n", ...)`:
+
+```go
+ if t.HoldsAwake {
+ // The box worked this out; we only pass it on. Worth a whole
+ // line rather than a marker on the row: the member's next
+ // question is "so what?", and the answer is what they are being
+ // billed for.
+ fmt.Fprintf(w, " %-*sthis fires more often than the idle window, so the instance never hibernates between ticks\n", nameW+2, "")
+ }
+```
+
+- [ ] **Step 4: Run the tests and watch them pass**
+
+Run: `TMPDIR=/tmp go test . -run TestScheduleLs -count=1 -v`
+Expected: PASS, both.
+
+- [ ] **Step 5: Whole-repo gate, then commit**
+
+```bash
+TMPDIR=/tmp go build ./... && TMPDIR=/tmp go vet ./... && TMPDIR=/tmp make test
+gofmt -l .
+git add schedule.go schedule_test.go
+git commit -m "cli: schedule ls says when a schedule holds its instance awake"
+```
+
+---
+
+### Task 7: Gate on real CI and open the PR
+
+**Files:** none.
+
+- [ ] **Step 1: Push to the CI remote**
+
+```bash
+git push amadan HEAD:james-hogging
+```
+
+- [ ] **Step 2: Read the run**
+
+`10-gate` is `go build ./...` then `make test`; `20-cross@5` adds darwin/windows/linux-arm64 compiles. Do not proceed until it is green — a branch pushed only to `origin` has been gated by nothing.
+
+- [ ] **Step 3: Confirm the change is inert, on paper**
+
+Before opening the PR, state in its body what the flagship will do after the roll and why nothing should visibly change: idle 15m, tightest schedule 5m, guard holds. Name the two apps to watch — `jam.bac.oncarlos.com` must stay resident, `seapointish.bag.oncarlos.com` must keep doing ~10 wake/sleep cycles a day. A change in either is a bug.
+
+- [ ] **Step 4: Open the PR**
+
+```bash
+git push origin james-hogging
+gh pr create --repo carlosframework/platform \
+ --title "Scheduled ticks must not hold an instance awake" \
+ --body "..." # spec link, the inert-on-roll argument, the two apps to watch
+```
+
+---
+
+## Self-Review
+
+**Spec coverage.** §3.1 → Tasks 2 and 4. §3.2 → Tasks 1 and 3. §3.3 → Tasks 5 and 6. §4 (inert on the flagship) → the Global Constraints and Task 7 Step 3. §5's exclusions are constraints, not tasks, and are listed in Global Constraints. §6's test list maps onto the tests in Tasks 2-5. §7 → Task 7.
+
+**Placeholders.** None. Every step carries the code it needs. Test helper names are explicitly flagged as stand-ins for whatever each file already provides, with the instruction to reuse rather than invent — that is a real instruction, not a deferral.
+
+**Type consistency.** `EarliestTimers() (map[string]int64, error)` is defined in Task 1, added to `DueStore` in Task 3, and called as `earliest[e.host]` there. `ServeScheduled` has the same four-argument signature as `ServeHTTP` in Tasks 2 and 4. `holdsAwake(registry.Timer, time.Duration, time.Time) bool` is defined and called with that order in Task 5. `HoldsAwake` is the field name in Tasks 5 and 6; `holds_awake` is the JSON tag in both.
+
+**One risk worth naming for the executor:** Task 2 renames a function on the hot path of every request (`beginRequest` → `begin` with a wrapper). If the package has other callers of `beginRequest` or `releaseFunc` beyond the ones named here, update them to the wrapper rather than threading `stamp` further — only the schedule deliverer should ever pass false.
diff --git a/docs/superpowers/specs/2026-08-30-scheduled-ticks-and-hibernation-design.md b/docs/superpowers/specs/2026-08-30-scheduled-ticks-and-hibernation-design.md
new file mode 100644
index 000000000..bdd713859
--- /dev/null
+++ b/docs/superpowers/specs/2026-08-30-scheduled-ticks-and-hibernation-design.md
@@ -0,0 +1,315 @@
+# Scheduled ticks must not hold an instance awake
+
+**Date:** 2026-08-30
+**Status:** approved in principle (Paul, 2026-08-30, after the flagship
+measurement below). Two decisions were taken in the brainstorm and are marked
+**(ruling)**; everything else is marked **(decision)** and can be overturned
+cheaply.
+
+## 1. The problem, as found
+
+Asked whether one member was hogging resources on the flagship, the answer was
+no — and the way the answer was reached turned up a bug underneath it.
+
+`james@keymail.dev` (account `bac`, app `jam`) declared one schedule on
+2026-08-29: `sweep`, every 5 minutes, hitting `/jobs/sweep`. The handler
+answers `204` in **1 millisecond**. From the moment that schedule landed
+(2026-08-29 21:25Z) the instance has been resident **continuously** — present
+in every 60-second metrics sample since, 24 hours and counting, where the
+comparable tenant `seapointish` sleeps and wakes ten times a day.
+
+It is not the app's fault and it is not the schedule's fault. It is one line
+of accounting.
+
+Scheduled delivery is otherwise exactly right. `activatorDeliverer` sends the
+tick through the activator into the instance's unix socket, never through the
+public edge, and its doc comment states the property it was built for
+(`internal/host/schedule_wiring.go:216`):
+
+> a hibernating instance WAKES for its tick, exactly as it would for a
+> visitor, and stays awake for the tick's duration because the activator's
+> in-flight count holds it open. No separate keep-awake to get wrong.
+
+The tick reaches the instance through `Service.ServeHTTP`
+(`internal/activator/activator.go:1141`) — the same door the edge uses — which
+calls `beginRequest`, whose release stamps `e.lastActive = a.now()`
+(`activator.go:594`). The sweeper hibernates on
+`e.inflight == 0 && now.Sub(e.lastActive) >= a.cfg.Idle` (`activator.go:933`).
+
+`CARLOS_HIBERNATE_IDLE` defaults to 15m. The spec is 5m. The clock is reset
+three times before it can ever expire, so the sweeper's condition is
+unreachable for the life of the schedule. **Every tick buys fifteen minutes of
+residency for one millisecond of work.**
+
+### Why it is worth fixing rather than tolerating
+
+Three reasons, in ascending order of weight.
+
+**It bills.** `instance_hours` is a metered, priced quantity, and in
+rate-sheet mode it is the one term with no free-tier subtraction applied —
+compare `billable(...)` on egress, storage, apps and memory against the raw
+`meters[usage.MeterInstanceHours]` in `internal/economics/compute.go:120`, at
+`InstanceHourUSD` = $0.01. A 5-minute schedule accrues ~730 instance-hours a
+month: **$7.30 billed for roughly four seconds of work**.
+
+**It defeats the capacity model.** The 2026-08-30 sizing measurement put peak
+concurrent awake instances at 74 of the top 300 accounts on a 90s window. That
+number assumes instances sleep between requests. An app with any schedule
+tighter than the idle window never sleeps at all, so as schedules spread the
+model degrades toward "everything is always awake" — which is the assumption
+the r8g.2xlarge recommendation was chosen against.
+
+**It is invisible.** Nothing tells the member. `carlos schedule ls` reports
+what will fire and what fired last; nothing says "and this interval means your
+instance never hibernates". James could not have known, and neither could
+anyone reading the console.
+
+## 2. What is already right, and must stay right
+
+Worth stating, because two of these were nearly re-engineered before the code
+was read properly.
+
+**There are already two wake paths for scheduled work, and they are enough.**
+The runner polls `DueTimers` every 15 seconds and delivers, waking the
+instance through the activator; separately the sweeper reads `DueInstances`
+and wakes any sleeping host whose recorded due time has arrived
+(`activator.go:909-914`). Platform timers are covered by the first.
+
+**`NextDue` is therefore not the mechanism to change.** `GET /api/next-due`
+over the instance socket (`internal/activator/backend_exec.go:506`) asks the
+*app* about work the platform cannot see — its own in-process schedule. It is
+not the platform's timer index and should not become one. This spec adds no
+column, no `SetNextDue` change and no second waker. **(decision)**
+
+**The sidecar precedent already solves the accounting half.**
+`BeginSidecarRequest` (`activator.go:526`) passes traffic to an awake instance
+without registering activity — "no in-flight count, no lastActive stamp (the
+sidecar's own long-lived WebSocket must not hold its instance awake; that
+non-count is the whole reason the pair can sleep)". The shape needed here is
+that shape, differing in one respect: a tick must *wake* a sleeping instance,
+where a sidecar request is refused.
+
+## 3. The change
+
+Three parts. Together they are perhaps a hundred lines.
+
+### 3.1 A non-stamping, waking entry point
+
+Add `Service.ServeScheduled(w, r, rt, proxy) bool`, sibling to `ServeHTTP`,
+and a `beginScheduledRequest` behind it that:
+
+- wakes a sleeping instance, exactly as `beginRequest` does — the tick is the
+ wake source, and that must not change;
+- takes the in-flight count for the duration, so a tick is never cut off
+ mid-delivery by a concurrent sweep;
+- on release, decrements **without** stamping `lastActive`.
+
+`activatorDeliverer.Deliver` calls it instead of `ServeHTTP`
+(`schedule_wiring.go:317`).
+
+**No wire marker. (decision)** The sidecar identifies itself with
+`SidecarHeader` because it has to: it dials its instance's public URL and its
+traffic arrives through the edge, so the only place to carry the signal is the
+request. A scheduled tick never leaves the process — "the tick never leaves
+the box: no DNS, no TLS, no edge auth, nothing an outsider could aim at the
+same path" (`schedule_wiring.go:219`) — so the call site itself is the signal.
+A dedicated method has no spoofing surface at all, where a header, even one
+that is safe by construction, would have to be argued about.
+
+`X-Carlos-Schedule-At` is untouched. It is the app-facing half of the tick
+contract and continues to reach the handler.
+
+### 3.2 The guard: don't sleep into an imminent tick
+
+Without a guard, part 3.1 alone converts `jam` from one resident process into
+**288 wake/sleep cycles a day**. Each cycle is a full SQLite
+checkpoint-and-restore traversal, and the wake path is where
+`2026-08-15-sqlite-corruption-defence-design.md` (issue #200) records an
+unclosed integrity gap. Multiplying a scheduled app's traversals by ~28× to
+save 26 MB is a bad trade.
+
+So: in `sweepOnce`, before hibernating an otherwise-idle host, skip the sleep
+when this box holds a timer for that host due sooner than `cfg.Idle` away.
+
+```
+hibernate when: inflight == 0
+ && now - lastActive >= cfg.Idle
+ && (no timer for this host, or nextAt - now > cfg.Idle)
+```
+
+**The threshold is `cfg.Idle` itself, not a new constant. (ruling — Paul,
+2026-08-30)** If the gap to the next tick is shorter than the window we would
+have stayed awake for anyway, staying awake *is* the cheaper answer. It adds
+no second tuning number to drift out of step with the first, and it composes
+with the idle window rather than fighting it:
+
+| Idle window | Schedule | Behaviour |
+|---|---|---|
+| 15m | every 5m | stays resident (today's behaviour, unchanged) |
+| 90s | every 5m | sleeps ~3.5 of every 5 minutes |
+| 90s | every 60s | stays resident |
+| 15m | every 1h | sleeps ~45 minutes |
+
+Residency is measured from each WAKE, not from each tick: a tick that wakes a
+sleeping instance stamps `lastActive` via the wake path's own stamp, so the
+instance gets one full idle window of residency starting at that wake before
+it becomes eligible to sleep again. That is why the 15m/1h row sleeps ~45
+minutes (60 minus the 15-minute window it earns on waking), not 59 (60 minus
+the one-minute tick that would apply if the clock started at delivery). Rows
+2 and 3 were already computed this way; anyone extending this table — or the
+§1 billing arithmetic that reads off it — should keep using the wake, not the
+tick, as the start of the window.
+
+**Implementation constraint, from the code as it stands:** `sweepOnce` does
+its `DueInstances` read deliberately before taking `a.mu` — "it's a registry
+read, and nothing under a.mu should wait on SQLite" (`activator.go:911`). The
+earliest-due lookup joins that pre-lock batch as one query returning
+`host → earliest next_at`, never a per-entry query inside the loop.
+
+A host with no timers is unaffected: no row, no guard, sleep as today.
+
+**A tick-held host still converges.** The guard only skips the *sleep*; it
+must not skip the build-convergence check that runs after it. A host held
+awake by the guard never reaches a "next wake" to converge on — that is what
+the guard guarantees — so it is now in the same class as an always-awake
+sidecar host, and reaches the convergence arm on every sweep exactly as a
+sidecar does. Without this, a `carlos ship` behind a tight enough schedule
+would be served stale indefinitely. **(ruling — Paul, review round 1,
+2026-08-31, overruling an earlier "converges at its next genuine sleep"
+draft of this section that assumed a sleep the guard makes impossible.)**
+
+### 3.3 Tell the member
+
+The box is the only place that knows both numbers — it holds the timer rows
+and it constructed the activator with `cfg.Idle`. So the box decides and
+reports.
+
+Add `HoldsAwake bool \`json:"holds_awake"\`` to `schedule.TimerStatus`
+(`internal/schedule/status.go:24`) — true when this row's interval is shorter
+than the box's idle window, so the row's instance stays resident between
+ticks. Written
+wherever the status object is already written (after every fire and every
+converge that changed a host's rows). Additive, per the repo's wire-shape
+rule: anything written before this change must still parse, and a status
+object from an older box simply lacks the field.
+
+`carlos schedule ls` renders it — one line, in the member's own vocabulary:
+that this schedule fires more often than the deployment's idle window, so the
+instance stays awake between ticks and accrues instance-hours accordingly.
+`carlos schedule set` shows the same line once the converger has written the
+status.
+
+**Warn, never refuse. (decision)** A tight schedule is a legitimate thing to
+want, and the platform serves it correctly today. The defect is that it was
+silent, not that it was allowed.
+
+## 4. The consequence to be explicit about
+
+**Not every host is inert under this change — it depends which side of the
+idle window the host's schedule sits on.**
+
+A host whose schedule fires *more often* than the idle window — `jam`, 5m
+against the flagship's 15m window — is unaffected. The guard holds it
+resident for the same reason the accounting bug used to: its next tick is
+always inside the window, so the sweeper's answer is "stay awake" today and
+after this lands. `jam` will not begin sleeping when this ships.
+
+A host whose schedule fires *less often* than the idle window is not inert,
+if that host also takes visitor traffic. Before this branch, every tick —
+regardless of how far apart — reset `lastActive`, so a tick landing between
+two visits kept pushing hibernation out. After this branch a tick no longer
+resets the clock, so that host loses the extra runway its ticks used to buy
+it: it can now hibernate up to one full idle window earlier than before,
+measured from its last real visitor request rather than from whichever tick
+last happened to land. That is the fix working as designed — a tick was
+never meant to count as activity — but it is a real, observable behaviour
+change on *this* branch, not a case of "provably alters nothing". Whether the
+flagship carries any host in this shape today is exactly what §7's rollout
+check exists to establish before rolling.
+
+The sequencing that remains deliberate is narrower than "inert, then
+behaviour change": this change runs under every tenant instance on every
+box, so shipping it and confirming its effect matches §7's expectations
+before separately tightening `CARLOS_HIBERNATE_IDLE` 15m → 90s keeps the two
+changes distinguishable if something needs reverting. The idle-window
+tightening is the change that turns `jam` from "stays resident" into
+"sleeps and wakes"; this branch is the change that stops loose-schedule
+hosts getting undeserved extra runway from their ticks.
+
+**(ruling — Paul, 2026-08-30)** The two land as separate branches.
+
+## 5. What is not built
+
+- **No `NextDue` rework, no new registry column, no second waker.** §2.
+- **No per-route idle override.** One knob per deployment, plus a guard that
+ reads it, covers every case found. A per-route window is a product decision
+ nobody has asked for.
+- **No new tuning constant.** §3.2.
+- **No special case for `carlos schedule run`.** A manual fire is
+ `FireTimerNow`, which moves `next_at` and nothing else
+ (`internal/registry/timers.go:132`); the runner cannot tell it from a
+ recurring fire without a new column. A schema change to buy "the instance
+ stays up 15 minutes after you press run" is not worth it. Manual runs are
+ ticks. **(decision — reversed during the brainstorm, after reading the
+ code: the first instinct was to treat a human's press as visitor activity.)**
+- **No use of `routes.next_due` in the guard.** It looks like free extra
+ coverage — the app's own declared due time, already in the registry, already
+ read by `DueInstances` — and it is a trap. That column is written only at
+ sleep and is deliberately not cleared on wake: "the registry's stale past
+ value while the host is up is harmless" (`activator.go:1007`). A guard
+ reading it on an awake host would compare against a timestamp in the past,
+ conclude the next delivery is imminent forever, and wedge itself open. The
+ value could only be made current by probing the app on every sweep pass,
+ which is a socket round trip per idle host per tick. The guard covers
+ platform timers; app-declared schedules stay uncovered, and are fine there
+ because their intervals (hourly and up, in every case on the fleet today)
+ are far wider than any plausible idle window.
+- **No change to sidecar accounting.** It is already correct and is the
+ precedent this follows.
+
+## 6. Testing
+
+All of it runs without a box.
+
+`internal/activator`:
+
+- a scheduled delivery leaves `lastActive` untouched; a visitor request to the
+ same host moves it — the two asserted against each other in one test, since
+ the bug is precisely that they were the same path;
+- a tick against a *sleeping* host wakes it, and the in-flight count holds it
+ through delivery (a sweep racing the tick must not stop it mid-flight);
+- the guard: a host idle past the window with a timer due inside `cfg.Idle`
+ stays resident; the same host with the timer beyond `cfg.Idle` hibernates;
+ a host with no timer rows hibernates as before;
+- the guard's query runs before `a.mu` — asserted structurally, the way the
+ existing sweep tests pin their ordering.
+
+`internal/host`:
+
+- `activatorDeliverer.Deliver` routes through `ServeScheduled`, not
+ `ServeHTTP`, and a delivery to a host that was asleep when the tick came due
+ still returns the handler's status;
+- `X-Carlos-Schedule-At` still reaches the handler unchanged.
+
+`internal/schedule`:
+
+- a status object written by a box that computed the verdict round-trips, and
+ one written without the field still parses.
+
+## 7. Rollout
+
+Ordinary: merge, roll the fleet, confirm the change lands where it should
+and nowhere else — see §4, this is not a blanket "confirm inert". Two checks:
+
+- **`jam`-shaped hosts — schedule interval shorter than the box's idle
+ window — must not move at all.** `jam` itself must still be resident
+ afterwards, and `seapointish` (no schedule) must still be doing its ten
+ wake/sleep cycles a day. A change to either here is a bug, not a win.
+- **Before rolling, enumerate hosts whose schedule interval is longer than
+ the idle window.** Expect their wake/sleep cycle counts to rise slightly —
+ this branch stops their ticks buying free runway between real visits, so a
+ small rise there is the change working as designed. Only a large or
+ otherwise unexplained jump is worth investigating.
+
+The separate idle-window tightening (15m → 90s) is verified on the *next*
+branch, against the same apps.
diff --git a/internal/activator/activator.go b/internal/activator/activator.go
index 609ccf743..426b8c5e9 100644
--- a/internal/activator/activator.go
+++ b/internal/activator/activator.go
@@ -173,6 +173,9 @@ type Backend interface {
type DueStore interface {
SetNextDue(host string, due int64) error
DueInstances(now int64) ([]registry.Route, error)
+ // EarliestTimers is the hibernation guard's input: host -> soonest armed
+ // timer. Read with DueInstances, before the sweep takes a.mu.
+ EarliestTimers() (map[string]int64, error)
}
// Lease is a held-but-not-yet-replicating lease (wake in progress; Eleven's
@@ -430,10 +433,15 @@ func New(cfg Config, backend Backend, due DueStore) *Service {
return a
}
-// beginRequest ensures the host's instance is awake and registers one in-flight
-// request against it. The returned release must be called when the request
-// finishes (it stamps last-activity, which starts the idle clock).
-func (a *Service) beginRequest(ctx context.Context, rt registry.Route) (release func(), err error) {
+// begin registers a request against rt's instance, waking it if asleep.
+//
+// stamp is what separates a visitor from the platform's own clock. Both take
+// the in-flight count — a delivery in progress must never be cut off by a
+// concurrent sweep — but only a visitor's release moves lastActive. A
+// scheduled tick that stamped would reset the idle window on every fire, so
+// any schedule tighter than that window would pin its instance awake forever
+// for however little work the handler does.
+func (a *Service) begin(ctx context.Context, rt registry.Route, stamp bool) (release func(), err error) {
deadline := time.NewTimer(wakeWaitBound)
defer deadline.Stop()
for {
@@ -475,7 +483,7 @@ func (a *Service) beginRequest(ctx context.Context, rt registry.Route) (release
}
e.inflight++
a.mu.Unlock()
- return a.releaseFunc(e), nil
+ return a.releaseFunc(e, stamp), nil
case actWaking:
ready := e.ready
@@ -510,6 +518,11 @@ func (a *Service) beginRequest(ctx context.Context, rt registry.Route) (release
}
}
+// beginRequest is begin for member traffic: activity counts.
+func (a *Service) beginRequest(ctx context.Context, rt registry.Route) (release func(), err error) {
+ return a.begin(ctx, rt, true)
+}
+
// errSidecarAsleep: a sidecar-marked request arrived for a host with no awake
// instance. Sidecar traffic never wakes anything — the activator itself spawns
// the sidecar as part of a wake, so a marked request for a sleeping host is
@@ -585,13 +598,15 @@ func (a *Service) BeginSidecarRequest(ctx context.Context, rt registry.Route) er
// proxying it away would only trade a short wait for a loud lease race) —
// and awakeCount() for the fleet status page.
-func (a *Service) releaseFunc(e *actEntry) func() {
+func (a *Service) releaseFunc(e *actEntry, stamp bool) func() {
var once sync.Once
return func() {
once.Do(func() {
a.mu.Lock()
e.inflight--
- e.lastActive = a.now()
+ if stamp {
+ e.lastActive = a.now()
+ }
a.mu.Unlock()
})
}
@@ -779,6 +794,11 @@ func (a *Service) Owns(host string) bool {
return a.entries[host] != nil
}
+// IdleWindow is cfg.Idle, for the box-side reporters that must tell a member
+// whether their schedule is tight enough to hold the instance resident. Read
+// only; the window is fixed at construction from CARLOS_HIBERNATE_IDLE.
+func (a *Service) IdleWindow() time.Duration { return a.cfg.Idle }
+
// Bounce initiates a graceful stop of host's RUNNING instance so that
// its next wake — the next request, or its schedule — respawns it with
// a freshly composed environment. This is config delivery's restart
@@ -914,6 +934,21 @@ func (a *Service) sweepOnce() {
if err != nil {
log.Printf("activator: due-time sweep: %v", err)
}
+ // The guard's input, read here for the same reason the due query is:
+ // nothing under a.mu may wait on SQLite. A failure does NOT degrade to
+ // today's behaviour: earliest ends up nil, so no host is held, and a
+ // host with a schedule tighter than the idle window flips from one
+ // resident process into a wake/sleep pair every tick — a 5m schedule
+ // becomes ~288 checkpoint-and-restore traversals a day, precisely the
+ // trade the guard exists to refuse. Fail open anyway: refusing to
+ // hibernate anything until the query recovers would hold every
+ // instance on the box awake, which is worse than losing the guard for
+ // the instances that have one.
+ earliest, err := a.due.EarliestTimers()
+ if err != nil {
+ log.Printf("activator: earliest-timer sweep: %v — hibernating without the schedule guard", err)
+ earliest = nil
+ }
a.mu.Lock()
cycled := 0 // build-convergence cycles started this tick (rate-limited)
for _, e := range a.entries {
@@ -931,9 +966,23 @@ func (a *Service) sweepOnce() {
continue
}
if e.inflight == 0 && now.Sub(e.lastActive) >= a.cfg.Idle {
- log.Printf("activator: %s idle %s — hibernating", e.host, now.Sub(e.lastActive).Truncate(time.Second))
- a.beginStopLocked(e)
- continue
+ // Don't sleep into an imminent tick. A wake/sleep pair is a full
+ // checkpoint-and-restore round trip; spending one to be absent
+ // for less time than we would have idled anyway is a straight
+ // loss. The threshold is the idle window itself rather than a
+ // constant of its own: it is already the deployment's stated
+ // opinion about how long residency is worth holding.
+ next, ok := earliest[e.host]
+ heldForTick := ok && time.Unix(next, 0).Sub(now) <= a.cfg.Idle
+ if !heldForTick {
+ log.Printf("activator: %s idle %s — hibernating", e.host, now.Sub(e.lastActive).Truncate(time.Second))
+ a.beginStopLocked(e)
+ continue
+ }
+ // Fall through to the convergence check below: a host held awake
+ // for its schedule never reaches a "next wake" to converge on, so
+ // it is now one of the always-awake hosts that arm was written
+ // for — the same class as a sidecar host that never sleeps.
}
// Converge onto a shipped build. A running child never re-reads
// live/<host>, so once a ship repoints the symlink this awake child is
@@ -1139,6 +1188,23 @@ func (a *Service) Shutdown(ctx context.Context) {
// it); false means "not activator-managed — proxy straight", so the edge can
// call it unconditionally.
func (a *Service) ServeHTTP(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
+ return a.serve(w, r, rt, proxy, true)
+}
+
+// ServeScheduled is ServeHTTP for a platform-delivered scheduled tick: same
+// wake, same in-flight hold, same error mapping — no activity credit.
+//
+// A dedicated method rather than a wire marker like SidecarHeader, and the
+// difference is the delivery path, not taste. The sidecar dials its
+// instance's public URL, so its traffic arrives through the edge and the
+// request is the only place a signal could ride. A tick never leaves the
+// process (schedule_wiring.go: "the tick never leaves the box"), so the call
+// site IS the signal, and there is no header for an outsider to send.
+func (a *Service) ServeScheduled(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
+ return a.serve(w, r, rt, proxy, false)
+}
+
+func (a *Service) serve(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler, stamp bool) bool {
// Per-host gating: only a hibernate-flagged instance is activator-managed;
// unflagged instances and sibling services are the edge's to proxy
// straight — the deploy-safety property that lets the bucket env ship
@@ -1181,7 +1247,7 @@ func (a *Service) ServeHTTP(w http.ResponseWriter, r *http.Request, rt registry.
// entry stayed local — placement converges at the next sleep boundary,
// since its state is already in S3. Single-box v1 always wakes locally.
- release, err := a.beginRequest(r.Context(), rt)
+ release, err := a.begin(r.Context(), rt, stamp)
if err != nil {
var held *errLeaseHeld
switch {
diff --git a/internal/activator/activator_test.go b/internal/activator/activator_test.go
index 7d0b9a92c..5893ae761 100644
--- a/internal/activator/activator_test.go
+++ b/internal/activator/activator_test.go
@@ -383,6 +383,39 @@ func newTestServiceAwake(t *testing.T, host string) *Service {
return a
}
+// arrangeAwake drives rt's host to actAwake on an already-built Service via a
+// real beginRequest/release round trip — the same arrange block
+// newTestServiceAwake uses, but against a caller-supplied Service/route
+// instead of building its own.
+func arrangeAwake(t *testing.T, a *Service, rt registry.Route) {
+ t.Helper()
+ release, err := a.beginRequest(context.Background(), rt)
+ if err != nil {
+ t.Fatalf("beginRequest %s: %v", rt.Host, err)
+ }
+ release()
+}
+
+// okHandler is a proxy stand-in that writes 200 OK — for tests asserting
+// only that a request reached the handler, not what a real instance would
+// return.
+func okHandler() http.Handler {
+ return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+ w.WriteHeader(http.StatusOK)
+ })
+}
+
+// armTimer puts one armed schedule row on host, due at next.
+func armTimer(t *testing.T, reg *registry.Registry, host string, next time.Time) {
+ t.Helper()
+ if err := reg.UpsertTimer(registry.Timer{
+ Host: host, Name: "sweep", Kind: "schedule", Spec: "every 5m",
+ Path: "/jobs/sweep", NextAt: next.Unix(), DueAt: next.Unix(),
+ }); err != nil {
+ t.Fatalf("UpsertTimer %s: %v", host, err)
+ }
+}
+
// waitFor polls until a condition holds — the stop path runs on its own
// goroutine, so tests observe its completion rather than assuming it.
func waitFor(t *testing.T, what string, cond func() bool) {
@@ -1332,6 +1365,226 @@ func TestServeHTTPUnmanagedRouteReturnsFalse(t *testing.T) {
}
}
+// A scheduled tick must not move lastActive: it is the platform's own clock
+// poking the instance, not a visitor. A tick that counted would make any
+// schedule tighter than the idle window a permanent keep-awake.
+func TestServeScheduledDoesNotStampActivity(t *testing.T) {
+ const host = "tick.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, time.Hour)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt) // a real beginRequest round trip, then release
+
+ a.mu.Lock()
+ before := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ clk.advance(time.Minute)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "http://carlos/jobs/sweep", nil)
+ if !a.ServeScheduled(rec, req, rt, okHandler()) {
+ t.Fatal("ServeScheduled returned false for a managed route")
+ }
+
+ a.mu.Lock()
+ after := a.entries[rt.Host].lastActive
+ inflight := a.entries[rt.Host].inflight
+ a.mu.Unlock()
+
+ if !after.Equal(before) {
+ t.Errorf("lastActive moved %v -> %v; a scheduled tick must not count as activity", before, after)
+ }
+ if inflight != 0 {
+ t.Errorf("inflight = %d after delivery, want 0 (the hold is released even though the stamp is not written)", inflight)
+ }
+}
+
+// The other half of the same coin, asserted against the first so the two
+// paths can never quietly become one again — which is the bug this fixes.
+func TestServeHTTPStillStampsActivity(t *testing.T) {
+ const host = "visitor.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, time.Hour)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ a.mu.Lock()
+ before := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ clk.advance(time.Minute)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("GET", "http://carlos/", nil)
+ if !a.ServeHTTP(rec, req, rt, okHandler()) {
+ t.Fatal("ServeHTTP returned false for a managed route")
+ }
+
+ a.mu.Lock()
+ after := a.entries[rt.Host].lastActive
+ a.mu.Unlock()
+
+ if !after.After(before) {
+ t.Errorf("lastActive did not move on a visitor request (%v -> %v)", before, after)
+ }
+}
+
+// A tick is a wake source and must stay one: an instance asleep when its
+// timer comes due has to be woken by the delivery itself.
+func TestServeScheduledWakesASleepingInstance(t *testing.T) {
+ const host = "asleep.bab.oncarlos.com"
+ a, _, _, _ := testActivator(t, time.Hour) // no entries yet: the host is asleep
+ rt := testRoute(host)
+
+ rec := httptest.NewRecorder()
+ req := httptest.NewRequest("POST", "http://carlos/jobs/sweep", nil)
+ if !a.ServeScheduled(rec, req, rt, okHandler()) {
+ t.Fatal("ServeScheduled returned false for a managed route")
+ }
+ if rec.Code != http.StatusOK {
+ t.Fatalf("status %d, want 200 — the tick must wake the instance, not be refused", rec.Code)
+ }
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("instance is not awake after a scheduled delivery")
+ }
+}
+
+// The guard: a host whose next platform tick is nearer than the idle window
+// stays resident. Sleeping would buy a few minutes of absence at the price of
+// a full checkpoint-and-restore round trip, and then an immediate wake.
+func TestSweepKeepsHostAwakeWhenATickIsImminent(t *testing.T) {
+ const host = "imminent.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // Arm 21 minutes out, then advance the clock 16 minutes: at sweep time
+ // the tick is 5 minutes in the FUTURE — genuinely inside the window, not
+ // merely overdue (that boundary is TestSweepKeepsHostAwakeForAnOverdueTick's
+ // job). A guard that only checked next <= now would wrongly hibernate here.
+ armTimer(t, reg, host, clk.now().Add(21*time.Minute))
+ clk.advance(16 * time.Minute) // lastActive is now well past the window
+
+ a.sweepOnce()
+
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("host hibernated with a tick due inside the idle window")
+ }
+}
+
+// Beyond the window, the sleep is worth taking.
+func TestSweepHibernatesWhenTheNextTickIsFarOff(t *testing.T) {
+ const host = "faroff.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // Next tick in an hour: well beyond the window.
+ armTimer(t, reg, host, clk.now().Add(time.Hour))
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ waitFor(t, "the host to hibernate with its next tick an hour away", func() bool {
+ a.mu.Lock()
+ defer a.mu.Unlock()
+ return a.entries[rt.Host] == nil
+ })
+}
+
+// A host with no timer rows is untouched by the guard — the overwhelming
+// majority of instances, and the case that must not regress.
+func TestSweepHibernatesAHostWithNoTimers(t *testing.T) {
+ const host = "untimed.bab.oncarlos.com"
+ a, _, clk, _ := testActivator(t, 15*time.Minute) // no timer rows at all
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ waitFor(t, "a host with no timers to hibernate as it always did", func() bool {
+ a.mu.Lock()
+ defer a.mu.Unlock()
+ return a.entries[rt.Host] == nil
+ })
+}
+
+// An overdue row (next_at already in the past — the runner has not got to it
+// yet) is the most imminent tick there is, so it holds the host too.
+func TestSweepKeepsHostAwakeForAnOverdueTick(t *testing.T) {
+ const host = "overdue.bab.oncarlos.com"
+ a, _, clk, reg := testActivator(t, 15*time.Minute)
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // next_at already behind us: the runner will fire it within 15s.
+ armTimer(t, reg, host, clk.now().Add(-time.Minute))
+ clk.advance(16 * time.Minute)
+
+ a.sweepOnce()
+
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("host hibernated with an overdue tick pending")
+ }
+}
+
+// A host held awake by the guard never reaches a "next wake" to converge on
+// — the same gap the always-awake sidecar case closes. The guard's continue
+// must fall through to the convergence check, not skip it, or a `carlos
+// ship` behind a tick-held host is served stale indefinitely.
+func TestSweepHeldForTickStillConverges(t *testing.T) {
+ const host = "tickheld.bab.oncarlos.com"
+ a, b, clk, reg := testActivator(t, 15*time.Minute)
+ var ver atomic.Value // the live/<host> symlink target; swapped to model a ship
+ ver.Store("v1")
+ a.liveVer = func(string) (string, error) { return ver.Load().(string), nil }
+ b.execVerFn = func() string { return ver.Load().(string) }
+
+ rt := testRoute(host)
+ arrangeAwake(t, a, rt)
+
+ // A tight schedule: next tick always inside the window, so the guard
+ // holds the host awake on every sweep from here on (an overdue row is
+ // still "inside the window" — more so, if anything).
+ armTimer(t, reg, host, clk.now().Add(5*time.Minute))
+ clk.advance(16 * time.Minute) // idle timeout reached
+
+ a.sweepOnce()
+ a.mu.Lock()
+ e := a.entries[rt.Host]
+ a.mu.Unlock()
+ if e == nil || e.state != actAwake {
+ t.Fatal("host hibernated despite an imminent tick — guard not exercised")
+ }
+
+ // Ship: repoint the symlink while the guard still holds the host awake.
+ ver.Store("v2")
+ a.sweepOnce() // starts the grace clock only
+ if a.entryCount() != 1 {
+ t.Fatal("cycled inside the grace window")
+ }
+
+ // Past the grace window: the guard still holds (the tick is now even
+ // more overdue), but convergence must still fire — that's the fix.
+ clk.advance(ConvergeGrace + time.Second)
+ a.sweepOnce()
+ waitFor(t, "a tick-held host to still converge onto the new build", func() bool {
+ return a.entryCount() == 0
+ })
+ if !b.lastProc().isStopped() {
+ t.Fatal("stale instance's process was not stopped")
+ }
+}
+
// A reload that finds an awake host no longer hibernate-flagged retires its
// child — stopped, final-checkpointed (replication closed), entry forgotten —
// the edge-process half of flipping hibernation off. And once the flag is
diff --git a/internal/host/schedule_wake_test.go b/internal/host/schedule_wake_test.go
index f5ea89ece..ce1ee966d 100644
--- a/internal/host/schedule_wake_test.go
+++ b/internal/host/schedule_wake_test.go
@@ -16,6 +16,7 @@ import (
"errors"
"net"
"net/http"
+ "net/http/httptest"
"net/http/httputil"
"path/filepath"
"strconv"
@@ -234,7 +235,7 @@ func TestScheduledTickWakesASleepingInstanceAndItHibernatesAgain(t *testing.T) {
w.WriteHeader(http.StatusNoContent)
})
- now := time.Date(2026, 8, 23, 13, 7, 0, 0, time.UTC)
+ now := time.Now().UTC()
occurrence := now.Unix() - 5
r := &schedule.Runner{
Reg: reg, Deliver: &activatorDeliverer{svc: svc}, Token: scheduleToken(dd),
@@ -530,3 +531,75 @@ func TestUnixProxyKeepsTheShapeCloseIdleNeeds(t *testing.T) {
t.Fatal("edge.UnixProxy's Transport is no longer an *http.Transport — closeIdle is now a no-op")
}
}
+
+// The bug, as a test: a schedule that fires more often than the idle window
+// used to pin its instance awake forever, because each tick stamped activity
+// and bought another full window. With the tick activity-invisible, the
+// instance sleeps between ticks and wakes again for the next one.
+//
+// No timer rows are armed here, deliberately: this exercises the delivery
+// leg alone. The sweep guard (which reads the timers table) would correctly
+// hold a host awake for a tick due this soon, and that is Task 3's test.
+func TestDeliveryDoesNotHoldTheInstanceAwake(t *testing.T) {
+ const host = "ticky.bab.oncarlos.com"
+ const idle = 300 * time.Millisecond
+ reg, inst, svc, _ := wakeFixture(t, host, idle, func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(http.StatusNoContent)
+ })
+ rt, err := reg.Get(host)
+ if err != nil {
+ t.Fatalf("get route: %v", err)
+ }
+ d := &activatorDeliverer{svc: svc}
+
+ // Tick twice per idle window for a few windows. Under the old accounting
+ // the instance could never reach the sweep's threshold; now it does,
+ // between ticks, and the next tick wakes it again.
+ deadline := time.Now().Add(20 * time.Second)
+ for inst.startCount() < 2 && time.Now().Before(deadline) {
+ req := httptest.NewRequest(http.MethodPost, "http://carlos/jobs/sweep", nil)
+ if _, err := d.Deliver(context.Background(), *rt, req); err != nil {
+ t.Fatalf("Deliver: %v", err)
+ }
+ time.Sleep(idle / 2)
+ }
+ if n := inst.startCount(); n < 2 {
+ t.Fatalf("the instance started %d time(s): it never hibernated between ticks, so the tick is still buying residency", n)
+ }
+}
+
+// The app-facing half of the tick contract is untouched by any of this.
+func TestDeliveryStillCarriesScheduleAtHeader(t *testing.T) {
+ const host = "hdr.bab.oncarlos.com"
+ seen := make(chan string, 1)
+ reg, _, svc, _ := wakeFixture(t, host, time.Hour, func(w http.ResponseWriter, r *http.Request) {
+ select {
+ case seen <- r.Header.Get("X-Carlos-Schedule-At"):
+ default:
+ }
+ w.WriteHeader(http.StatusNoContent)
+ })
+ rt, err := reg.Get(host)
+ if err != nil {
+ t.Fatalf("get route: %v", err)
+ }
+ d := &activatorDeliverer{svc: svc}
+
+ req := httptest.NewRequest(http.MethodPost, "http://carlos/jobs/sweep", nil)
+ req.Header.Set("X-Carlos-Schedule-At", "1788126300")
+ status, err := d.Deliver(context.Background(), *rt, req)
+ if err != nil {
+ t.Fatalf("Deliver: %v", err)
+ }
+ if status != http.StatusNoContent {
+ t.Fatalf("status = %d, want 204", status)
+ }
+ select {
+ case got := <-seen:
+ if got != "1788126300" {
+ t.Errorf("handler saw X-Carlos-Schedule-At = %q, want it passed through unchanged", got)
+ }
+ case <-time.After(10 * time.Second):
+ t.Fatal("the handler never ran")
+ }
+}
diff --git a/internal/host/schedule_wiring.go b/internal/host/schedule_wiring.go
index a0af2ac4a..266f3a434 100644
--- a/internal/host/schedule_wiring.go
+++ b/internal/host/schedule_wiring.go
@@ -73,6 +73,13 @@ func newScheduleDeps(reg *registry.Registry, store release.Store, svc *activator
sink.Event(release.RouteAccount(rt), rt.App, rt.Host, level, msg)
}
}
+ // Zero when no activator is armed on this box: nothing hibernates, so no
+ // schedule can hold anything awake, and holdsAwake reports false for all
+ // of them.
+ var idle time.Duration
+ if svc != nil {
+ idle = svc.IdleWindow()
+ }
sd := &scheduleDeps{
runner: &schedule.Runner{
Reg: reg,
@@ -81,6 +88,7 @@ func newScheduleDeps(reg *registry.Registry, store release.Store, svc *activator
Event: event,
Box: box,
Build: version.Build,
+ Idle: idle,
},
}
if store == nil {
@@ -98,6 +106,7 @@ func newScheduleDeps(reg *registry.Registry, store release.Store, svc *activator
}
}
sd.conv = schedule.NewConverger(reg, store, box, report)
+ sd.conv.Idle = idle
sd.runner.Report = report
return sd
}
@@ -214,8 +223,16 @@ func scheduleLegFor(sd *scheduleDeps) func(ctx context.Context) {
// Two properties come from that and only that:
//
// - a hibernating instance WAKES for its tick, exactly as it would for a
-// visitor, and stays awake for the tick's duration because the activator's
-// in-flight count holds it open. No separate keep-awake to get wrong.
+// visitor, and is held awake for the tick's duration because the
+// activator's in-flight count holds it open — but the tick earns no
+// activity credit, so it does not by itself keep the instance awake
+// afterwards. Separately, the sweeper's schedule guard (activator.go,
+// sweepOnce) holds a host resident whenever its next tick is due sooner
+// than the idle window, so a schedule firing more often than the idle
+// window stays resident by that guard's design rather than thrashing
+// through a wake/sleep pair on every tick. A schedule looser than the
+// idle window is not covered by the guard and does sleep between ticks,
+// waking again for the next one.
// - the tick never leaves the box: no DNS, no TLS, no edge auth, nothing an
// outsider could aim at the same path.
type activatorDeliverer struct {
@@ -324,10 +341,12 @@ func (d *activatorDeliverer) Deliver(ctx context.Context, rt registry.Route, req
}
proxy := d.proxyFor(rt.Socket)
h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
- // ServeHTTP reports false for a route it does not manage (not an
- // instance, or not hibernate-flagged); that route is proxied straight,
- // the same call shape the edge itself uses.
- if d.svc == nil || !d.svc.ServeHTTP(w, r, rt, proxy) {
+ // ServeScheduled, not ServeHTTP: a tick wakes the instance and is
+ // held in flight for its duration, but must not reset the idle
+ // clock. Reports false for a route it does not manage (not an
+ // instance, or not hibernate-flagged); that route is proxied
+ // straight, the same call shape the edge itself uses.
+ if d.svc == nil || !d.svc.ServeScheduled(w, r, rt, proxy) {
proxy.ServeHTTP(w, r)
}
})
diff --git a/internal/registry/timers.go b/internal/registry/timers.go
index 0d3bba0ef..12cdde238 100644
--- a/internal/registry/timers.go
+++ b/internal/registry/timers.go
@@ -214,3 +214,30 @@ func (r *Registry) DeleteTimersExcept(host, kind string, keep map[string]bool) (
}
return int(n), nil
}
+
+// EarliestTimers is the hibernation guard's batch: for every host with at
+// least one armed row, the soonest next_at among them. One query, because
+// the activator's sweep reads it BEFORE taking its mutex (activator.go's
+// "nothing under a.mu should wait on SQLite") and a per-entry query inside
+// the sweep loop would put SQLite exactly where that comment forbids it.
+//
+// next_at = 0 is "not armed" — a parked cron, or a row converge wrote and
+// the runner has not yet dated — and is excluded here rather than by the
+// caller, matching DueTimers' own `next_at > 0` gate.
+func (r *Registry) EarliestTimers() (map[string]int64, error) {
+ rows, err := r.sql.Query(`SELECT host, MIN(next_at) FROM timers WHERE next_at > 0 GROUP BY host`)
+ if err != nil {
+ return nil, err
+ }
+ defer rows.Close()
+ out := make(map[string]int64)
+ for rows.Next() {
+ var host string
+ var next int64
+ if err := rows.Scan(&host, &next); err != nil {
+ return nil, err
+ }
+ out[host] = next
+ }
+ return out, rows.Err()
+}
diff --git a/internal/registry/timers_test.go b/internal/registry/timers_test.go
index 3e68a127d..57b2eb85c 100644
--- a/internal/registry/timers_test.go
+++ b/internal/registry/timers_test.go
@@ -5,6 +5,7 @@ import (
"errors"
"os"
"path/filepath"
+ "reflect"
"testing"
)
@@ -249,3 +250,41 @@ func TestDueTimersExcludesBusyHostsBeforeTheLimit(t *testing.T) {
t.Fatalf("all hosts excluded = %+v, want none", due)
}
}
+
+func TestEarliestTimersPerHost(t *testing.T) {
+ r := openTestRegistry(t)
+ rows := []Timer{
+ // Two rows for one host: the earliest wins.
+ {Host: "a.example", Name: "late", Kind: "schedule", Spec: "every 1h", Path: "/x", NextAt: 5000, DueAt: 5000},
+ {Host: "a.example", Name: "soon", Kind: "schedule", Spec: "every 5m", Path: "/y", NextAt: 1200, DueAt: 1200},
+ // A second host is reported independently.
+ {Host: "b.example", Name: "only", Kind: "schedule", Spec: "every 5m", Path: "/z", NextAt: 3000, DueAt: 3000},
+ // next_at == 0 means "not armed" and must never appear.
+ {Host: "c.example", Name: "parked", Kind: "schedule", Spec: "every 5m", Path: "/p", NextAt: 0, DueAt: 0},
+ }
+ for _, tm := range rows {
+ if err := r.UpsertTimer(tm); err != nil {
+ t.Fatalf("UpsertTimer %s/%s: %v", tm.Host, tm.Name, err)
+ }
+ }
+
+ got, err := r.EarliestTimers()
+ if err != nil {
+ t.Fatalf("EarliestTimers: %v", err)
+ }
+ want := map[string]int64{"a.example": 1200, "b.example": 3000}
+ if !reflect.DeepEqual(got, want) {
+ t.Errorf("EarliestTimers = %v, want %v", got, want)
+ }
+}
+
+func TestEarliestTimersEmpty(t *testing.T) {
+ r := openTestRegistry(t)
+ got, err := r.EarliestTimers()
+ if err != nil {
+ t.Fatalf("EarliestTimers: %v", err)
+ }
+ if len(got) != 0 {
+ t.Errorf("EarliestTimers on an empty table = %v, want empty", got)
+ }
+}
diff --git a/internal/schedule/control.go b/internal/schedule/control.go
index 6a0b7cf0a..14e663f5f 100644
--- a/internal/schedule/control.go
+++ b/internal/schedule/control.go
@@ -122,7 +122,7 @@ func Mount(mux *http.ServeMux, host string, store TimerStore, token func() (stri
http.Error(w, "registry error", http.StatusInternalServerError)
return
}
- writeTimer(w, *row)
+ writeTimer(w, *row, now())
}))
mux.HandleFunc("DELETE /schedule/{name}", authed(func(w http.ResponseWriter, r *http.Request) {
@@ -156,7 +156,11 @@ func Mount(mux *http.ServeMux, host string, store TimerStore, token func() (stri
Timers []TimerStatus `json:"timers"`
}{Timers: []TimerStatus{}}
for _, row := range rows {
- out.Timers = append(out.Timers, toStatus(row))
+ // The control socket has no view of the activator's idle window
+ // (it is reached from inside the exec child, not the box's
+ // hibernation wiring), so it makes no holds_awake claim — same
+ // as any box that predates the field.
+ out.Timers = append(out.Timers, toStatus(row, 0, now()))
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(out)
@@ -164,8 +168,10 @@ func Mount(mux *http.ServeMux, host string, store TimerStore, token func() (stri
}
// writeTimer writes one registry row as its wire TimerStatus (toStatus is
-// converge.go's — one row-to-status mapping for the whole package).
-func writeTimer(w http.ResponseWriter, r registry.Timer) {
+// converge.go's — one row-to-status mapping for the whole package). idle is
+// 0 here too, for the reason the /schedule GET handler above documents: the
+// control socket cannot see the activator's window.
+func writeTimer(w http.ResponseWriter, r registry.Timer, now time.Time) {
w.Header().Set("Content-Type", "application/json")
- json.NewEncoder(w).Encode(toStatus(r))
+ json.NewEncoder(w).Encode(toStatus(r, 0, now))
}
diff --git a/internal/schedule/converge.go b/internal/schedule/converge.go
index c17df56bd..adbfca188 100644
--- a/internal/schedule/converge.go
+++ b/internal/schedule/converge.go
@@ -34,6 +34,7 @@ type Converger struct {
Now func() time.Time
Box string // reported in Status.Box
Report func(rt registry.Route, st Status) // called for every host whose rows changed or that is unsupported; nil = no reports
+ Idle time.Duration // the activator's idle window; zero when no hibernation is armed on this box
declEtag map[[2]string]string // (acct, app) → last etag seen
declDoc map[[2]string]Declaration
@@ -171,7 +172,7 @@ func (c *Converger) convergeApp(ctx context.Context, acct, app string, rts []reg
continue
}
if changed && c.Report != nil {
- st, err := StatusFor(c.Reg, rt, c.Box, now)
+ st, err := StatusFor(c.Reg, rt, c.Box, now, c.Idle)
if err != nil {
errs = append(errs, err)
continue
@@ -309,19 +310,31 @@ const UnsupportedUnit = "unit-backed instance — scheduled work delivers to exe
// refuses these at `set`; this is the defence for one that slipped past it).
const StatusNeverFires = "never fires"
-// StatusFor builds the host's report from its rows.
-func StatusFor(reg *registry.Registry, rt registry.Route, box string, now time.Time) (Status, error) {
+// StatusFor builds the host's report from its rows. idle is the activator's
+// idle window on this box, threaded through to holdsAwake — zero when no
+// hibernation is armed on the box at all.
+//
+// Hibernation is also armed PER ROUTE (rt.Hibernate — activator.go will not
+// sweep a host unless Hibernate && Status == "ready"), so a box-wide window
+// is not sufficient on its own: a route with Hibernate false was never
+// going to sleep, schedule or no schedule, and holds_awake must say so by
+// reporting false for every one of its rows, not by naming a tight timer
+// that changes nothing for it.
+func StatusFor(reg *registry.Registry, rt registry.Route, box string, now time.Time, idle time.Duration) (Status, error) {
st := Status{Schema: 1, Host: rt.Host, Box: box, ReportedAt: now.UTC().Format(time.RFC3339), Timers: []TimerStatus{}}
if rt.Backing == registry.BackingUnit {
st.Unsupported = UnsupportedUnit
return st, nil
}
+ if !rt.Hibernate {
+ idle = 0
+ }
rows, err := reg.TimersFor(rt.Host)
if err != nil {
return st, err
}
for _, r := range rows {
- st.Timers = append(st.Timers, toStatus(r))
+ st.Timers = append(st.Timers, toStatus(r, idle, now))
}
return st, nil
}
@@ -329,6 +342,48 @@ func StatusFor(reg *registry.Registry, rt registry.Route, box string, now time.T
// toStatus maps one timers row onto its wire shape. Task 5 (the `carlos
// schedule ls` reader / the runner's own report path) reuses this so the
// row-to-status mapping is written once.
-func toStatus(t registry.Timer) TimerStatus {
- return TimerStatus{Name: t.Name, Kind: t.Kind, Spec: t.Spec, Path: t.Path, NextAt: t.NextAt, DueAt: t.DueAt, LastAt: t.LastAt, LastStatus: t.LastStatus, LastMs: t.LastMs, Attempts: t.Attempts}
+func toStatus(t registry.Timer, idle time.Duration, now time.Time) TimerStatus {
+ return TimerStatus{
+ Name: t.Name, Kind: t.Kind, Spec: t.Spec, Path: t.Path,
+ NextAt: t.NextAt, DueAt: t.DueAt, LastAt: t.LastAt,
+ LastStatus: t.LastStatus, LastMs: t.LastMs, Attempts: t.Attempts,
+ HoldsAwake: holdsAwake(t, idle, now),
+ }
+}
+
+// holdsAwake reports whether t's interval is tight enough that its instance
+// never gets to sleep: the gap between consecutive occurrences is no wider
+// than idle, so the activator's guard (and, before it, the tick's own wake)
+// keeps the process resident indefinitely.
+//
+// The gap comes from two Next calls rather than Spec.Every, so a cron gets a
+// real answer instead of a special case — but for an irregular cron (e.g.
+// "0,5 * * * *": five minutes then fifty-five) that first gap is a SAMPLE,
+// not a bound: the same row can answer true at :59 and false at :01,
+// depending on when the report happens to be built. That is the direction
+// to err in — over-warning about a bill beats missing one — and is chosen
+// deliberately, not an oversight.
+//
+// idle <= 0 means hibernation is not armed — either the box has no
+// activator, or (StatusFor) the route itself has Hibernate off — so
+// nothing holds anything awake, because nothing sleeps. A once-timer fires
+// once and cannot hold anything. An unparseable spec is not evidence of a
+// problem, so it makes no claim.
+func holdsAwake(t registry.Timer, idle time.Duration, now time.Time) bool {
+ if idle <= 0 || t.Kind != KindSchedule {
+ return false
+ }
+ sp, err := ParseSpec(t.Spec)
+ if err != nil {
+ return false
+ }
+ first := sp.Next(now)
+ if first.IsZero() {
+ return false
+ }
+ second := sp.Next(first)
+ if second.IsZero() {
+ return false
+ }
+ return second.Sub(first) <= idle
}
diff --git a/internal/schedule/converge_test.go b/internal/schedule/converge_test.go
index e451c2273..c8375bf39 100644
--- a/internal/schedule/converge_test.go
+++ b/internal/schedule/converge_test.go
@@ -11,6 +11,62 @@ import (
"github.com/carlosframework/platform/internal/release"
)
+func TestHoldsAwakeWhenTheIntervalIsInsideTheIdleWindow(t *testing.T) {
+ now := time.Unix(1788120000, 0).UTC()
+ cases := []struct {
+ name string
+ kind string
+ spec string
+ idle time.Duration
+ want bool
+ }{
+ {"tighter than the window", KindSchedule, "every 5m", 15 * time.Minute, true},
+ {"exactly the window", KindSchedule, "every 15m", 15 * time.Minute, true},
+ {"wider than the window", KindSchedule, "every 1h", 15 * time.Minute, false},
+ {"tight, short window", KindSchedule, "every 5m", 90 * time.Second, false},
+ {"a once-timer never holds", KindOnce, "at 1788130000", 15 * time.Minute, false},
+ {"hibernation not armed", KindSchedule, "every 1m", 0, false},
+ {"hourly cron", KindSchedule, "cron 0 * * * *", 15 * time.Minute, false},
+ {"every-minute cron", KindSchedule, "cron * * * * *", 15 * time.Minute, true},
+ {"unparseable spec is not a claim", KindSchedule, "banana", 15 * time.Minute, false},
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ got := holdsAwake(registry.Timer{Kind: tc.kind, Spec: tc.spec}, tc.idle, now)
+ if got != tc.want {
+ t.Errorf("holdsAwake(%q, idle=%s) = %v, want %v", tc.spec, tc.idle, got, tc.want)
+ }
+ })
+ }
+}
+
+// Hibernation is armed PER ROUTE (rt.Hibernate), not just per box: a route
+// with Hibernate off was never going to sleep, so no schedule on it can
+// hold anything awake, however tight. StatusFor must zero the window before
+// holdsAwake ever sees it — a box-wide idle window alone is not enough.
+func TestStatusForNeverClaimsHoldsAwakeOffARouteWithHibernateOff(t *testing.T) {
+ reg, err := registry.Open(filepath.Join(t.TempDir(), "r.db"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { reg.Close() })
+ rt := registry.Route{Host: "resident.bab.oncarlos.com", Kind: "instance", Account: "bab", App: "app", Status: "ready", Hibernate: false}
+ if err := reg.Upsert(rt); err != nil {
+ t.Fatal(err)
+ }
+ now := at("2026-08-23T13:07:00Z")
+ if err := reg.UpsertTimer(registry.Timer{Host: rt.Host, Name: "sweep", Kind: KindSchedule, Spec: "every 5m", Path: "/jobs/sweep", CreatedAt: now.Unix()}); err != nil {
+ t.Fatal(err)
+ }
+ st, err := StatusFor(reg, rt, "box1", now, 15*time.Minute)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(st.Timers) != 1 || st.Timers[0].HoldsAwake {
+ t.Fatalf("HoldsAwake = %v on a Hibernate:false route, want false regardless of the box's idle window", st.Timers[0].HoldsAwake)
+ }
+}
+
func convergeFixture(t *testing.T) (*registry.Registry, *release.DirStore) {
t.Helper()
reg, err := registry.Open(filepath.Join(t.TempDir(), "r.db"))
diff --git a/internal/schedule/runner.go b/internal/schedule/runner.go
index e1e6501b8..e4b5c1e7b 100644
--- a/internal/schedule/runner.go
+++ b/internal/schedule/runner.go
@@ -60,6 +60,7 @@ type Runner struct {
Report func(rt registry.Route, st Status)
Box string
Build string
+ Idle time.Duration // the activator's idle window; zero when no hibernation is armed on this box
// Per-host work is DETACHED from the tick: Tick launches one goroutine
// per host with due rows and returns without waiting for any of them.
@@ -176,7 +177,7 @@ func (r *Runner) Tick(ctx context.Context) {
// A report that cannot be built is a silent hole in `carlos
// schedule ls` — the member's only view of this box. Say so:
// the fires themselves still landed on the rows.
- if st, err := StatusFor(r.Reg, rt, r.Box, r.now()); err != nil {
+ if st, err := StatusFor(r.Reg, rt, r.Box, r.now(), r.Idle); err != nil {
log.Printf("schedule: %s: build status report: %v", rt.Host, err)
} else {
r.Report(rt, st)
diff --git a/internal/schedule/status.go b/internal/schedule/status.go
index b70b352a3..cc51aa6f6 100644
--- a/internal/schedule/status.go
+++ b/internal/schedule/status.go
@@ -32,6 +32,13 @@ type TimerStatus struct {
LastStatus string `json:"last_status"`
LastMs int64 `json:"last_ms"`
Attempts int `json:"attempts"`
+
+ // HoldsAwake is the box's warning: this row fires at least as often as
+ // the deployment's idle window, so its instance never hibernates between
+ // ticks and accrues instance-hours around the clock. Computed by the box,
+ // which is the only place that holds both numbers. False from a box that
+ // predates the field, and false wherever hibernation is not armed.
+ HoldsAwake bool `json:"holds_awake,omitempty"`
}
// Status is apps/<acct>/<app>/schedule-status/<host>.json (spec §4.3):
diff --git a/internal/schedule/status_test.go b/internal/schedule/status_test.go
index e6d3c8e60..ac1f472e7 100644
--- a/internal/schedule/status_test.go
+++ b/internal/schedule/status_test.go
@@ -2,11 +2,25 @@ package schedule
import (
"context"
+ "encoding/json"
"testing"
"github.com/carlosframework/platform/internal/release"
)
+// The wire shape is additive: an object written before this field existed
+// must still parse, and read back as false.
+func TestStatusWithoutHoldsAwakeStillParses(t *testing.T) {
+ raw := []byte(`{"schema":1,"host":"h","box":"b","reported_at":"2026-08-30T21:40:05Z","timers":[{"name":"sweep","kind":"schedule","spec":"every 5m","path":"/jobs/sweep","next_at":1788126300}]}`)
+ var st Status
+ if err := json.Unmarshal(raw, &st); err != nil {
+ t.Fatalf("unmarshal a pre-holds_awake status: %v", err)
+ }
+ if len(st.Timers) != 1 || st.Timers[0].HoldsAwake {
+ t.Errorf("HoldsAwake = %v, want false for an object written before the field existed", st.Timers[0].HoldsAwake)
+ }
+}
+
func TestWriteAndListStatus(t *testing.T) {
ctx := context.Background()
st := release.NewDirStore(t.TempDir())
diff --git a/schedule.go b/schedule.go
index 172a7db42..8d4af1f44 100644
--- a/schedule.go
+++ b/schedule.go
@@ -351,6 +351,13 @@ func renderScheduleInstance(w io.Writer, inst schedule.Status, now time.Time) {
last += " (" + t.LastStatus + ")"
}
fmt.Fprintf(w, " %-*snext %s last %s\n", nameW+2, labels[i], next, last)
+ if t.HoldsAwake {
+ // The box worked this out; we only pass it on. Worth a whole
+ // line rather than a marker on the row: the member's next
+ // question is "so what?", and the answer is what they are being
+ // billed for.
+ fmt.Fprintf(w, " %-*sthis fires more often than the idle window, so the instance never hibernates between ticks\n", nameW+2, "")
+ }
}
}
diff --git a/schedule_test.go b/schedule_test.go
index baeda5a9b..0fbac0a45 100644
--- a/schedule_test.go
+++ b/schedule_test.go
@@ -6,6 +6,7 @@ package main
// one client-side refusal (`-every 30s`) that must never reach the network.
import (
+ "bytes"
"context"
"encoding/json"
"net/http"
@@ -311,6 +312,48 @@ func TestRenderScheduleListNilSlices(t *testing.T) {
}
}
+func TestScheduleLsFlagsAHoldsAwakeRow(t *testing.T) {
+ var b bytes.Buffer
+ now := time.Unix(1788126000, 0).UTC()
+ inst := schedule.Status{
+ Schema: 1, Host: "jam.bac.oncarlos.com", Box: "annie-welding",
+ ReportedAt: now.Format(time.RFC3339),
+ Timers: []schedule.TimerStatus{{
+ Name: "sweep", Kind: schedule.KindSchedule, Spec: "every 5m",
+ Path: "/jobs/sweep", NextAt: now.Add(2 * time.Minute).Unix(),
+ LastAt: now.Add(-3 * time.Minute).Unix(), LastStatus: "204", LastMs: 1,
+ HoldsAwake: true,
+ }},
+ }
+ renderScheduleInstance(&b, inst, now)
+
+ out := b.String()
+ if !strings.Contains(out, "sweep") {
+ t.Fatalf("row missing from output:\n%s", out)
+ }
+ if !strings.Contains(out, "never hibernates") {
+ t.Errorf("no holds-awake warning in output:\n%s", out)
+ }
+}
+
+func TestScheduleLsSaysNothingForAnOrdinaryRow(t *testing.T) {
+ var b bytes.Buffer
+ now := time.Unix(1788126000, 0).UTC()
+ inst := schedule.Status{
+ Schema: 1, Host: "seapointish.bag.oncarlos.com", Box: "annie-welding",
+ ReportedAt: now.Format(time.RFC3339),
+ Timers: []schedule.TimerStatus{{
+ Name: "sync", Kind: schedule.KindSchedule, Spec: "every 6h",
+ Path: "/jobs/sync", NextAt: now.Add(time.Hour).Unix(),
+ }},
+ }
+ renderScheduleInstance(&b, inst, now)
+
+ if strings.Contains(b.String(), "never hibernates") {
+ t.Errorf("warned about a 6-hourly schedule:\n%s", b.String())
+ }
+}
+
// TestScheduleVerbRefusals drives the real binary for the ways `carlos
// schedule` stops before it ever dials anything: direct bucket mode, and
// -every 30s (schedule.ValidEvery's "whole minutes only" rule) — run with