carlos / platform Public

secrets set and env set land a file the running process never reads, and nothing anywhere says so open

Filing this as a silent-failure surface rather than a design disagreement. The design is coherent and I am not asking for it to change.

What happens

carlos secrets set --environment relay succeeds, the merged config is materialized to /data/connect.go.tito.io.config.env, and carlos secrets list confirms delivery. The running process has not read any of it, and will not until somebody restarts it by hand.

Nothing in the loop says so. secrets set reports success, secrets list reports the key, and the app behaves exactly as it did before. On our side the symptom is an OAuth provider that stays invisible with no error, because LoadApps reads an unset pair as "not offered".

Why this is not just our problem to remember

We already have it written down twice in our own runbook, in bold, with "nothing will remind you" beside it. That we needed to write it twice is the report: a step that is mandatory, invisible, and has no failure signal is one people will keep missing, and each of us will miss it independently.

Our own record of the reasoning is:

> carlos routes to these processes without spawning or owning them, so the > restart half belongs to the app's own supervisor.

That is right, and it is why this is not a request to have CARLOS restart things. A TCP-addr exec route's lifecycle is the app's. But there is a gap between "CARLOS does not own the restart" and "CARLOS delivers a file and has no way to mention it", and the gap is where the silence lives.

What would close it, cheapest first

  1. Say it in the output. secrets set and env set already know the route is TCP-addr exec. One line — "delivered; this route's process will not see it until it restarts" — costs nothing and removes the whole failure class.
  2. A post-delivery hook on the route. Optional, app-declared, run after materialization. The app supplies the restart, which keeps the ownership line exactly where it is.
  3. Something to ask. A way to see which delivered generation a route's process is actually running, so a verification step exists that is not "grep the app's own startup log for a count it happens to print".

(1) alone would have saved us the two runbook entries.

Adjacent

#13 ("carlos deploy reports live while the old binary is still serving") is the same family — the platform reporting a state that describes the adopted artifact rather than the running process — but a different mechanism. This one is env and secret delivery rather than deploy.

Small aside from the same corner

carlos secrets set stores an empty value without complaint. Ours arrive as "$(op read ...)", and op read on a field that does not exist returns nothing, so a typo in a field name delivers two empty strings and reports success. With both halves empty the provider is silently not offered; with one empty the relay refuses to boot. Refusing an empty value, or naming it, would catch that at the point it happens. Happy to file separately if it is a different conversation.

Correcting my own framing, because I checked after filing and the runbook I based it on is out of date.

There IS a restart verb. carlos restart — "cycle an app's instances — no new version, no config change". So "CARLOS will not restart it for you" is wrong as stated, and our own runbook says it twice. That is on us and I am fixing it there.

What is actually missing is granularity. carlos restart's only selector is --app. For us that app is titogo, and carlos routes lists nineteen routes under it, every one KIND=instance:

  • the relay (connect.go.tito.io, environment relay)
  • the router (platform.ie.go.tito.io, environment router)
  • the tunnel
  • every customer instance (bill.go.tito.io, tito.go.tito.io, …)
  • every canary

So the verb that exists would bounce all of production to pick up one credential delivered to one route's bundle. I have not run it to confirm the blast radius, for the obvious reason.

That makes the ask smaller and more concrete than what I filed:

carlos restart --host <h> (or --route), cycling one route rather than the app. Everything else in the original still stands — most of all point (1), that secrets set could simply say the delivery will not be seen until the route restarts, which costs nothing and removes the whole failure class.

The --environment angle might be even better fit for this case: a bundle is delivered to an environment (relay, router, demo), and "restart what reads this bundle" is exactly the intent after a secrets set --environment. That would make the two verbs symmetrical, which is how I would expect to reach for them.

Apologies for the noise in the original framing. The gap is real, it is just narrower than I said.

Picked this up and landed the first ask. b8fd86c on main.

What shipped

(1), the output line. env set, env unset, secrets set, secrets unset and env exec-delivery now name the routes a write reached that nothing will bounce:

set 2 secret(s) for titogo (environment relay)
note: connect.go.tito.io has the file but is still running the old values. carlos routes to this process without owning it. Restart to fix.
(this machine's routes only)

Where carlos does know the unit that owns the process — an addr route with Unit set, issue #110's shape — it offers the command instead:

note: console.amadan.net has the file but is still running the old values. carlos restart --app titogo cycles console.service.

That split matters more than it looks. For the plain shape, the one you reported, carlos restart reaches nothing at all: the agent's leg skips the route and the activator has no child for a process it never spawned. Pointing you at that command would have been a second silent failure answering the first.

The aside, on both sides of the seal. secrets set FOO= is refused now, naming every empty key at once. It is also refused at the console door, which turned out to be possible after all — AES-GCM is length-preserving, so an envelope at exactly the floor sealed nothing, and the console can see that without holding a key. That means an older or hand-rolled client cannot get an empty secret past it either.

Three states, not two

The part of your report that shaped the design most was the runbook detail — written down twice, in bold. A notice that can only say "these routes" or nothing would have recreated the same trap one layer up, so it distinguishes:

  • routes delivered to,
  • routes it could not determine (their bundle would not read, or the bundle they are bound to is missing — that one resolves to an error, not to the app default, so nothing was delivered and saying "delivered" would be a lie),
  • and "no claim was made", for a console predating the field or a laptop with no registry near it, which prints nothing rather than a "could not check" line on every write.

"Could not tell" never renders as "all clear".

Two honest limits. The console reads its own box's registry, so a multi-box fleet's other boxes are not represented (#145's blind spot) — hence the caveat line. And a set to the same value still counts as a change, because a secret re-seals to a fresh envelope every time and only the box can compare plaintexts.

Also corrected

carlos env sync was telling you "exec-backed instances converge within seconds". True of the file, false of the program reading it, for exactly your route shape — and it is the command someone runs while waiting for a change to take effect. It now says the files land in seconds and anything carlos does not own keeps its old values until you restart it.

Not built

(2) the post-delivery hook and (3) the delivered-generation readback. Both are real and neither is in this change. (3) is the one that would give you a verification step rather than a reminder, and it is the harder half: the generation object already exists per app, but nothing records which generation a process actually loaded, which needs something on the app's side of the line. Worth its own thread if you want it.

#13 stays open and unaddressed here — same family, different mechanism, as you said.

Triage, 2026-09-05. Your narrowed ask is built — branch restart-selectors (6017525), pushed. CI is still running as I write this; read it before merging.

carlos restart --host connect.go.tito.io carlos restart --environment relay carlos restart --app titogo --dry-run # what the app-wide form would cycle

You were right that --environment is the better fit, and for the reason you gave: a bundle is delivered to an environment, so "restart what reads this bundle" is the intent after secrets set --environment relay. Both are in.

The shape, since it is the part worth checking

A narrowed request does not touch restart-gen.json. It goes to a sibling object, restart-scope.json, through a separate console door.

That is the safety property rather than tidiness. Boxes act on restart-gen.json's etag moving and never read its content — so a selector written into that object would be invisible to every box on today's binary, and each would read the movement the only way it knows how: cycle the whole app. The failure mode of getting this wrong is precisely the nineteen-route bounce you declined to run. Same hazard server-side: the existing /restart door reads no body, so a narrowed request sent there would be answered app-wide by the console.

So: an old box sees no movement on the object it watches and cycles nothing; an old console 404s and the CLI says so rather than retrying against the door that would widen; an unknown future selector kind is stepped over rather than widened. Never widened on ignorance is the rule throughout.

It is an append log rather than a generation because etag movement cannot carry which routes — two requests inside one 2s poll window would collapse, silently dropping a restart its caller was told had been requested.

Blast radius is now visible before you commit to it

--dry-run resolves the selector, prints the routes, writes nothing and bumps no serial. With no selector it answers the question you had and wouldn't run the verb to find out: what would --app titogo actually cycle. Non-dry narrowed runs name the addressed set too, including asleep — left as it is for hibernators, and carry the same "this machine's routes only" caveat your first fix introduced.

Hibernating instances are still never woken. --app alone is byte-for-byte what it was.

A selector matching nothing is refused, naming what the app does have; --host plus a mismatched --environment is refused rather than resolved, because one of the two is a typo. That is the same instinct as your original report: a step that silently does the wrong thing is worse than one that fails.

Not built, deliberately

Your (2) post-delivery hook and (3) delivered-generation readback. Both still open, and (3) is still the one that would give you verification instead of a reminder — it needs something on the app's side of the line, and it is the same missing piece #11 wants for an AWAKE column and #13 wants for a running-build signal. Three issues, one absent field. Worth deciding once.

One loose end from your own fix: secrets set's note still says carlos restart --app %s. That string could now point at --host. Not touched, because changing what those verbs print was out of scope here — but it is a one-line follow-up and it makes the pair symmetrical, which is how you said you'd reach for them.

Worth a human pass before this lands: the new user-facing strings have not been through copy-review (it needs an interactive session and this ran in the background). They follow existing in-repo phrasing, but this repo has a rule about that gate and I would rather say so than let it slide.