carlos / platform Public

Migrated from github.com/carlosframework/platform issue #95, opened 2026-08-07. GitHub issues are closed for this repo; see AGENTS.md, "Work is tracked on amadan, not GitHub".

Still outstanding: First sight of a new app's config etag triggers no convergence and no log line, so the first env set is silently absorbed until the hourly floor.

---

Summary

The first carlos env set for an app is silently absorbed by the fast config loop. Nothing converges until either a second write with a different value, or the hourly floor — up to an hour later. To the operator it looks like env set did nothing.

Observed on a live deployment: set BUILD, waited, page unchanged; set it again to a different value and it applied within seconds.

Cause

genPollPass (internal/host/agent.go) only acts on an app it has seen before:

prev, seen := cache[k]
cache[k] = etag
if seen && prev != etag {
    execChanged = execChanged || inExec[k]
    unitChanged = unitChanged || inUnit[k]
}

On first sight seen is false, so the generation etag is recorded and no convergence is triggered — whatever the value. The second write moves a known etag and fires immediately.

This is deliberate, and the reasoning in configGenLoop's doc comment is sound:

> No fast first pass. The loop sleeps BEFORE its first tick and the first > sight of every app is silent (see genPollPass) — boot convergence belongs > to configSyncLoop's ~1min first pass, and a box coming up must not bounce > every instance it has because it has never seen their etags before.

Not bouncing every instance on boot is obviously right. The problem is the case that rationale does not cover: an app the loop has never seen because it was created after the agent started, or whose first config write is also the operator's first. There is no boot storm to prevent there — there is exactly one app and one write.

Evidence

One config event all day, for the second write only:

14:03:48   apps/bab/echo/config.json written      (second env set)
14:03:50   echo.bab...: config updated
14:03:50   agent: config changed, bounced echo.bab...

The earlier write produced no config updated, no bounce, and no log line of any kind.

Why it is worse than a slow path

• It is silent. No "observed, deferred to the hourly pass" line. The operator has no way to tell "not applied yet" from "not applied".
• It lands on the first-run path. A new app's first env set is precisely the case that hits this, so it is most likely to be met by someone with least context.
• The workaround teaches the wrong lesson. Setting the value twice looks like superstition, and setting it to the same value a second time will not work either — the etag must actually move.

Suggested fixes

1. Treat first sight as a change when the app's route is new to this box — the boot-storm concern is about a box restarting with existing instances, not about an app appearing for the first time. If the distinction is hard to draw, (2) is enough.
2. Log the deferral. One line — "first sight of <app> config; deferring to the hourly pass" — turns an apparent no-op into an understood wait.
3. Have carlos env set say what happens next. It knows it wrote the object; it could print that exec-backed instances converge within seconds and unit-backed ones on the next root pass, or that a first write may wait for the hourly floor. Same shape as apps create printing the next command.

(2) alone would have saved the confusion here.

Aside, possibly its own issue

While looking at this, the same instance logs a repeating litestream warning every ~30s:

WARN timeout waiting for db initialization system=store level=3
  dbs=[/data/echo.bab.carlos.exciting.io.db] timeout=30s
  hint="database may have corrupted local state or blocked transactions..."

The app is stateless and never creates that file, so replication waits 30s for a database that will never exist, at four levels, forever. Related to #94 (--db is passed to every exec-backed instance whether or not the app has any use for one) — a stateless app appears to get replication set up for it regardless.