carlos / platform Public

Download this file

1	# The CLI
2
3	`carlos` is one binary that plays three roles, picked by the first word you
4	type. Most days you are in the first role: you have an app, and you want it
5	running somewhere. The second is for people who stand a deployment up and
6	administer it. The third is what systemd starts on a box.
7
8	Run `carlos help` for the same grouping at the terminal, and
9	`carlos <command> -h` for a command's own flags. This page says what each
10	command is for and when you would reach for it. It does not list every flag,
11	because `-h` already does.
12
13	Three flags turn up almost everywhere, and the sections below do not repeat
14	them. `--app` names the app (`-a` for short). `--account` names the account it belongs to, or
15	`--account-id` when you want a sqid matched as a sqid and nothing else; leave
16	both off if you belong to exactly one account. `--console` picks which
17	logged-in console to act through, and you need it only when this terminal is
18	logged in to more than one deployment.
19
20	Most app commands work two ways. Signed in with `carlos auth login`, they go
21	through a console's API, which is how someone with no box access and no cloud
22	credentials gets their work done. With `CARLOS_DEPLOYMENT_BUCKET` or
23	`CARLOS_DEPLOYMENT_DIR` set, they write to the bucket directly. That is the
24	operator's path. Where a command only works one way, its section says so.
25
26	## Describing changes
27
28	Add a short description to a mutation with `--message`. Commands sent through the console record it alongside the exact Activity operation and target.
29
30	```sh
31	carlos deploy --message "Fix invoice total rounding" ./app-linux-arm64
32	```
33
34	This applies to `ship`, `deploy`, `promote`, `rollback`, `restart`, `instances create`, `instances delete`, `features set`, `env set`, `env unset`, and `env exec-delivery`. A `deploy` command uses one description for both `ship` and `promote`.
35
36	In an interactive terminal, these commands ask for a description by default. Automation never receives a prompt. `restart --dry-run` also never prompts. Use `--no-prompt` to skip the question. To disable the default prompt, add this to `.carlos/config`:
37
38	```json
39	{"message_prompt":"off"}
40	```
41
42	An explicit `--message` still works with either setting. An empty description is valid. Messages must be valid UTF-8, contain no control characters such as line breaks or tabs, and be no longer than 240 Unicode characters. Never include secrets.
43
44	Release `--label` and release notes describe the release separately from a mutation message.
45
46	## Your apps
47
48	Everything from claiming a name to watching a deploy land. None of it needs
49	access to a box, a cloud console, or the deployment's bucket; that is a rule
50	the platform holds itself to, and a command that broke it would be a bug.
51
52	### carlos auth
53
54	Logs this terminal in to a console. `carlos auth login` prints a short code
55	that you approve in a browser already signed in to that console, and the
56	browser does not have to be on this machine, so this works over SSH. The
57	token lands in `~/.carlos/credentials` at mode 0600, one entry per console,
58	so a laptop can hold credentials for the flagship and your own deployment at
59	once.
60
61	```sh
62	carlos auth login
63	carlos auth whoami
64	```
65
66	Do not do this on a box. A box has no browser and no human, and the
67	long-lived token you would leave behind is a standing credential nobody is
68	watching; pass `--owner <address>` to
69	[carlos accounts](/docs/cli#carlos-accounts) instead. `carlos auth default`
70	shows the console this machine talks to by default, and sets it when you give
71	it a URL. With `--project` it writes `./.carlos/config`, which you can commit
72	so a checkout pins its own deployment. With `-d <name>` it shows or sets the
73	console of one [destination](/docs/cli#deploying-one-checkout-to-several-places).
74
75	### carlos config
76
77	Edits `.carlos/config` for you, so you never have to hand-edit its JSON.
78	`carlos config destinations` manages the named places that
79	[carlos deploy](/docs/cli#carlos-deploy) and every other command reach with
80	`-d <name>`:
81
82	```sh
83	carlos config destinations add production --console https://carlos.example.com --app shop
84	carlos config destinations add partner --console https://carlos.example.org --app shop
85	carlos config destinations
86	carlos config destinations default partner
87	carlos config destinations set partner --account me@example.org
88	carlos config destinations rename partner eu
89	carlos config destinations remove eu
90	```
91
92	`add` takes `--console`, `--account`, `--app`, `--kind`, `--artifact` and
93	`--target`, and `--default` makes the new destination the default. `set`
94	changes the same keys on an existing destination, and an empty value
95	(`--app ""`) removes a key. `list`, which is also what the bare command does,
96	marks the destination used when none is named with `*`.
97
98	Each verb edits the nearest `.carlos/config` in this directory or above it,
99	and creates `./.carlos/config` when there is none. `--global` edits
100	`~/.carlos/config` instead. `add` warns when this machine is not logged in to
101	the destination's console, and still writes the destination, so you can set
102	up a console before you log in to it.
103
104	These commands never let a file quietly start deploying somewhere new. If the
105	file already names a console, account or app at the top, adding the first
106	destination moves those keys into a destination of their own, which becomes
107	the default. At a terminal, `add` asks what to call it; elsewhere you name it
108	with `--from-current <name>`. When a second destination is added, the first
109	one is recorded as the default. Removing the default is refused while more
110	than one other destination remains.
111
112	### carlos apps
113
114	Claims an app name, moves an app onto a customer fleet, and deletes or
115	restores one. Creating goes through the same door the console's New app form
116	uses, so a name that form would refuse is refused here too.
117
118	```sh
119	carlos apps create --app hello
120	```
121
122	`--region <code>` sets the region the app's instances start in. Without it,
123	the console picks the region nearest you from your timezone, and the command
124	prints one line saying which and why:
125
126	```sh
127	carlos apps create --app hello --region uk
128	```
129
130	A name is also refused when its platform apex, `<app>.<account>.<apps
131	domain>`, is already taken: by another app's instance, even one in the Trash,
132	or by any route there at all. The new app would otherwise never get an address
133	of its own. Renaming an app, or moving it to another account, is checked the
134	same way. The error names whatever holds the address. Remove that instance
135	first, or pick another name.
136
137	`carlos apps delete` moves an app to the Trash. Every route drops, custom
138	domains included, the name stays reserved, and you have thirty days to
139	`carlos apps restore` it, or three for a temporary app. Restoring brings back
140	releases, channels and the production flag exactly as they were, but no
141	routes at all, not even the platform apex, so plan on adding those again.
142	A temporary app is different: its instance records are kept in the Trash, so
143	its hosts put its routes back on their next pass after a restore, with nothing
144	to add by hand. Delete is refused while the app is flagged production. Clear
145	the flag first.
146
147	`--place fleet/<name>` at create, or `carlos apps place --target fleet/<name>`
148	later, puts the app's routes and instances on a fleet your account owns.
149	`--target ""` brings them back to platform-served. `--target` is mandatory even
150	in that empty spelling: leaving it off is a usage error, never a silent
151	clear.
152
153	Two more flags are stamped at claim time. `--control-plane` marks the app as
154	the control plane, after which it refuses trash, purge, rename and transfer,
155	and refuses a service credential's promote — owner only. `--build-probe /path`
156	names a path whose response body carries the running build, `/healthz` serving
157	`ok abc1234` being the shape; [carlos deploy](/docs/cli#carlos-deploy) then
158	polls that instead of the `X-Carlos-Version` header, which the edge stamps from
159	the route row and which therefore reports the new build before the process has
160	actually restarted. Both are checked against what the console echoes back, so a
161	write that silently failed is an error rather than a quiet lie.
162
163	`carlos apps build-probe` sets that same probe on an app that already exists,
164	which is the common case: most apps were claimed before anyone thought about
165	it.
166
167	```sh
168	carlos apps build-probe --app docs /api/version
169	```
170
171	Pass `-` instead of a path to clear it and go back to watching the header. It
172	is worth setting on anything you deploy often. Without it, a green deploy tells
173	you the release was adopted, not that the process running the old one ever
174	restarted — and those look identical from outside.
175
176	A temporary app is one `carlos deploy --temporary` created. When its time is
177	up it moves to the Trash, and it is purged 3 days later instead of 30.
178	`carlos apps ttl` moves its expiry, anywhere from an hour to 30 days from now.
179	`carlos apps keep` makes it permanent, and it keeps its `tmp-` name.
180
181	```sh
182	carlos apps list
183	carlos apps ttl --app tmp-docs 5d
184	carlos apps keep --app tmp-docs
185	```
186
187	`carlos apps set` changes how the app's running instances take a new build:
188
189	```sh
190	carlos apps set hello --swap hold
191	carlos apps set hello --swap-concurrency 3
192	```
193
194	With `--swap overlap`, the default, the new build starts beside the old one
195	and takes over once it answers its health check, so no request waits. With
196	`--swap hold`, the old one stops first, and requests wait until the new one
197	is up. `--swap-concurrency` is how many instances may swap at once on one
198	box, after the first one has stayed up for ten seconds; it starts at 1. Any
199	member of the app can change both, here or on the app's settings page in the
200	console.
201
202	`carlos apps set` also controls the update settings an instance may change:
203
204	```sh
205	carlos apps set shop --instance-control updates,pin --default-updates manual
206	```
207
208	--instance-control names the settings an instance may change itself once instances can act for themselves. --default-updates and --default-major apply wherever an instance has not chosen; --default-channel is the channel new instances are created on.
209
210	### carlos ship
211
212	Publishes a release. The artifact is hashed, stored under its content
213	address, and recorded in a manifest that never changes afterwards. Shipping
214	does not change what anything serves — for that, see
215	[carlos promote](/docs/cli#carlos-promote).
216
217	```sh
218	carlos ship --app hello --label "fixing modals" ./hello
219	```
220
221	`--version` is optional once the app has a version target set, and leaving it
222	off is the point of having one: ship mints the next iteration under it. Use
223	`--kind static` with a directory to ship a site instead of a binary. `--label`
224	is one line for humans and shows up in `carlos releases` and in the console,
225	so write it for the person doing a rollback at 2am.
226
227	### carlos promote
228
229	Points a channel at a version you have already shipped.
230
231	**Channels belong to the app, not to the platform.** A new app is born with
232	one channel — `edge` by default — and that one channel is its production.
233	More channels are opt-in, declared by a pipeline, and named whatever the app
234	calls them. So there is no universal ladder to memorise: `carlos channels`
235	lists what this app has, and `carlos pipeline` shows the order promotion must
236	follow through them.
237
238	Leave the channel off and an app with one channel promotes onto it. An app
239	with several says so, and names them, rather than guessing:
240
241	```sh
242	carlos promote --app hello a1b2c3d # the app's one channel
243	carlos promote --app hello a1b2c3d edge # or name it
244	```
245
246	An app that declares no pipeline keeps the frozen legacy ladder,
247	`canary → edge → beta → stable` — beta entered from edge and stable from
248	beta, so "it was signed off on beta" means something, and reaching stable
249	cuts the release's semver tag. Channels named `canary/<something>` are never
250	refused, because that is what the fleet itself runs on.
251
252	Promoting the same version twice succeeds and does nothing. `--hotfix` skips
253	the ladder and is an operator act: it works on the direct-bucket path only,
254	and the console refuses it by name, so the flag never quietly does nothing.
255
256	### carlos deploy
257
258	Ship, promote, and then wait until the app's URL actually serves the new
259	build. Reach for this one by default; ship and promote separately when you
260	want the gap between them.
261
262	Deploy then watches each box roll the build out. An awake instance counts
263	once the new build has passed its health check and taken over; on each box
264	the first one must also stay up for ten seconds before the rest follow. An
265	asleep instance counts once its box has installed the build, and starts it on
266	its next request. If the build fails on a box, that box stops rolling it out,
267	its other instances stay on the build they were serving, and deploy names the
268	failure and exits with an error.
269
270	Where a box does not report its rollout yet, deploy watches the app's URL
271	instead, and what it watches there is worth knowing, because the two answers
272	differ. If the app
273	declares a build probe, deploy asks the app's own process which build it is
274	running, and `live` means the process cycled. If it does not, deploy watches
275	the `X-Carlos-Version` header — and the edge stamps that from the route row
276	the moment adoption moves the symlink, whether or not anything restarted. So
277	a probe-less deploy can print `live` over a process still running the old
278	binary, and it says so while it waits. Set one with
279	[carlos apps build-probe](/docs/cli#carlos-apps); it is a single command per
280	app.
281
282	```sh
283	carlos deploy --app hello ./hello
284	```
285
286	Signed-in only. With saved project defaults you can run `carlos deploy` with
287	no arguments and no questions. A first run at a terminal fills the blanks by
288	asking. If your account owns no apps yet, it offers to claim one named after
289	the directory you are in, through the same door as `carlos apps create`. It
290	then asks which artifact to ship, works out from what you point it at whether
291	that is a binary or a site, and offers to save both answers as the project's
292	defaults. With neither arguments nor defaults and no terminal to ask at — CI,
293	usually — it refuses instead of guessing. `--channel` overrides the channel it
294	picks (normally the one the app's instances already follow), and `--host`
295	scopes the watch to a single instance. A static site needs that second flag,
296	having no instances to resolve a channel from.
297
298	```sh
299	carlos deploy --app website --kind static --host www.example.com ./dist
300	```
301
302	`--swap hold` ships a release that stops each instance before starting the
303	new build, for a migration the previous build cannot share the database
304	with. It applies to that release only; `carlos apps set --swap hold` makes it
305	the rule for the app.
306
307	To try a build that failed again, once you have fixed what it depended on,
308	promote it again with `--retry`. Every box where it failed tries it again:
309
310	```sh
311	carlos promote --app hello --retry a1b2c3d edge
312	```
313
314	Without `--retry`, promoting a build that is already on the channel does
315	nothing.
316
317	When `carlos deploy` creates your first app, it sends your timezone so the
318	console can pick the region nearest you, and prints the choice. `--region
319	<code>` chooses instead. On an app that already exists, `--region` must
320	match the app's region, or deploy stops before shipping anything.
321
322	`--temporary` creates a new app for a static site that cleans up after
323	itself. It is named `tmp-` plus six random letters and digits, or
324	`tmp-<name>` with `--app <name>`, and lives 7 days unless `--ttl` says
325	otherwise (1h to 30d). `--ttl` on its own implies `--temporary`. A temporary
326	app is served only at its own platform address, so `--host` and
327	`--kind binary` are refused with it, and an account holds at most 20 of them.
328
329	```sh
330	carlos deploy --temporary --ttl 3d ./dist
331	```
332
333	#### Redirects for a static site
334
335	Put a file named `_redirects` at the top of the site directory. It uses the
336	format Netlify and Cloudflare Pages use: one rule per line, with a source, a
337	destination and an optional status.
338
339	```text
340	# source destination status
341	/old-page /new-page 301
342	/blog/* /posts/:splat 301
343	/docs/:slug /guide/:slug 302
344	/app/* /app/index.html 200
345	https://www.example.com/* https://example.com/:splat 301!
346	/* /404.html 404
347	```
348
349	- A source is a path, or a URL with a host. A URL source applies only to
350	requests for that host, so the last-but-one rule sends `www` to the apex.
351	- `:name` matches one path segment. A `*` at the end matches the rest of the
352	path, and the destination gets it as `:splat`.
353	- The status is 301 when you leave it out. 302, 303, 307 and 308 are also
354	redirects. 200 serves the destination's file at the requested URL, which
355	is how a single-page app serves its client-side routes. 404 serves the
356	destination's file as the not-found page.
357	- A 200 or 404 rule takes a path in the same site. A redirect can also go to
358	another host.
359	- A request's query string is kept unless the destination has one of its
360	own.
361	- When a file exists at the requested path, the file wins. Add `!` to the
362	status (`301!`) to make the rule win.
363	- Rules apply in order, and the first rule that matches is used. A line
364	starting with `#` is a comment.
365
366	`carlos ship` and `carlos deploy` refuse a `_redirects` file with a line they
367	cannot use, and name the line. Netlify's conditions, such as `Country=`, and
368	proxying to another host are not supported. A file can have up to 2,000
369	rules. The edge does not serve the `_redirects` file itself.
370
371	#### Two builds at once
372
373	When a new build reaches an awake instance, it starts beside the old one,
374	and for a few seconds both run. They share the instance's database and its
375	scheduled work. The new build's migrations run while the old build is still
376	serving, so both builds must work with the database in between:
377
378	- Add in one release and remove in a later one. A new column, table or index
379	goes in one release; dropping the old one waits for the next.
380	- Keep the names of scheduled work (`carlos schedule`) the same across the
381	two builds.
382
383	When a release cannot follow these rules, ship it with `--swap hold`, or set
384	the app to `hold` with `carlos apps set`. Each instance then stops before the
385	new build starts, and requests wait until it is up.
386
387	#### Deploying one checkout to several places
388
389	When one project deploys to more than one console, or as more than one app,
390	name each place in `.carlos/config` under `destinations` and pick one with
391	`--destination` (`-d` for short):
392
393	```json
394	{
395	"default": "production",
396	"artifact": "./bin/server",
397	"destinations": {
398	"production": {"console": "https://carlos.example.com", "account": "me@example.com", "app": "shop"},
399	"partner": {"console": "https://carlos.example.org", "app": "shop", "target": "linux-amd64"}
400	}
401	}
402	```
403
404	```sh
405	carlos deploy -d partner
406	carlos logs -d partner
407	```
408
409	Each destination takes the same keys as the top of the file, and any key it
410	leaves out comes from the top of the file. Here both destinations ship
411	`./bin/server`. Every command that takes `--console` also takes
412	`--destination`, and `CARLOS_DESTINATION` does the same job for a whole
413	shell. With no destination named, the file's `default` applies. A file with
414	only one destination needs no default. A file with several destinations and
415	no default refuses to guess, and so does a destination name that no config
416	file defines. Saved answers go into the destination you are using.
417	[carlos config destinations](/docs/cli#carlos-config) adds, changes and
418	removes destinations, so you don't need to edit the file yourself.
419	`--target` is unrelated: it names the platform (GOOS-GOARCH) a binary was
420	built for.
421
422	### carlos canary
423
424	Deploy onto a host and a channel belonging to this branch alone, so nothing
425	the app already serves changes. It creates the canary's instance record if
426	it needs to, ships, promotes, and then waits on the canary URL's own
427	`X-Carlos-Version` — the same proof [carlos deploy](/docs/cli#carlos-deploy)
428	demands, pointed somewhere that cannot disturb production.
429
430	```sh
431	carlos canary --app jam ./jam
432	```
433
434	The name defaults to the current git branch, slugified; `--name` overrides
435	it. A branch called `new-queue` gives you the channel `canary/new-queue` and
436	the host `new-queue-jam.<your account>.<apps domain>`. Since the same string
437	has to be both a channel segment and a hostname label, an explicit `--name`
438	is checked rather than quietly repaired — lowercase letters, digits and
439	dashes. The name and the app share one hostname label, so together they can be at most 63 characters.
440
441	`--region <code>` places a new canary in that region (default: the app's
442	region). It is refused for an existing canary in another region, and for an
443	app that runs on one of your fleets.
444
445	Promotion into a canary is never refused, on any app, from any rung. Nor
446	does a canary cost you a rung: canary channels sit outside every pipeline, so
447	a build that has been on one still enters the pipeline as a fresh build.
448	Canary, then promote to your entry channel, and the ladder is untouched.
449
450	**If your app carries config, reach for `--environment`.** A canary is a
451	second instance standing alongside a live one, and config in CARLOS is
452	per-app: with no bundle of its own the canary serves the app's default one,
453	where every origin-shaped value names the app's *main* host. Anything keyed
454	to that single scalar — the `Origin` header the app compares against,
455	magic-link URLs, a WebAuthn RP ID, the cookie `Secure` flag — refuses
456	requests on the canary host, so the canary serves GETs perfectly and fails
457	every form submission. The verb says so when you leave the flag off.
458
459	```sh
460	carlos env --app jam --environment jam-canary JAM_ORIGIN=https://new-queue-jam.bab.oncarlos.com
461	carlos canary --app jam --environment jam-canary ./jam
462	```
463
464	```sh
465	carlos canary ls --app jam
466	carlos canary rm --app jam --name new-queue
467	```
468
469	`ls` shows each canary's host, what it is serving, which config bundle it is
470	bound to, and what is left of its seven-day lease — refreshed every time you re-promote, so an active canary
471	never expires and a forgotten one starts saying so. `rm` deletes the
472	instance record: the host stops being served and the process goes away. The
473	channel itself remains, followed by nothing, and lapses at the end of its
474	lease; there is no pointer to delete.
475
476	One thing this is not: traffic splitting. A canary is a URL, not a
477	percentage of production traffic on the app's own hostname.
478
479	### carlos restart
480
481	Cycles running processes with no new version and no config change. This is
482	the answer to a wedged instance.
483
484	```sh
485	carlos restart --app hello
486	```
487
488	It reports the restart as *requested*, which is the honest word: the console
489	touches one small object and every box serving the app notices within
490	seconds. Long-running tenants come back within seconds. Exec-backed instances
491	stop within seconds and respawn on their next request, which for an idle app
492	may be a while. Nothing is stuck: the instance comes back with the next
493	request that needs it. Hibernating instances are left asleep.
494
495	#### Restarting less than the whole app
496
497	An app is not always a small thing. One app can own a relay, a router, a
498	tunnel and every customer instance — so "restart the app" can mean cycling all
499	of production to pick up one credential that reached one route. Two flags
500	narrow it:
501
502	```sh
503	carlos restart --app titogo --host bill.example.com
504	carlos restart --app titogo --environment relay
505	```
506
507	`--host` cycles exactly that route. `--environment` cycles every route reading
508	that named config bundle, which is the one to reach for after a
509	[carlos secrets set](/docs/cli#carlos-secrets) or
510	[carlos env set](/docs/cli#carlos-env) on the same `--environment`: a bundle is
511	delivered to an environment, so "restart what reads this" is the request that
512	follows it.
513
514	Both narrowed forms print the routes they addressed — host, environment, the
515	unit that owns the process where there is one, and which of them were asleep
516	and so left alone. `--app` on its own still prints the single line it always
517	has; `--dry-run` is how you see its set.
518
519	```sh
520	carlos restart --app titogo --environment relay --dry-run
521	```
522
523	A dry run resolves the selector, names every route it would cycle, and stops
524	there. Nothing is written and nothing restarts.
525
526	Precedence, when you give both: `--host` wins, but only after the console
527	checks that the host really does read that environment. If it does not, the
528	command refuses rather than picking one — one of the two flags is a typo, and
529	this is not a verb to guess on. A selector that matches nothing is refused the
530	same way, and the refusal lists the hosts or environments the app actually
531	has. Nothing is ever widened back to the whole app.
532
533	Two limits worth knowing. The console resolves the selector against its own
534	box's registry, and it refuses a selector nothing there matches — so on a
535	multi-box fleet you can only ask for a `--host` or an `--environment` that at
536	least one route on the console's own box reads, and the printed set is what
537	that box can see. A request it does accept is a bucket object every box
538	converges on, and each box applies the same selector to its own routes; those
539	other routes are cycled without appearing in the list. And a console or a box
540	that predates these flags does not understand a narrowed request: the console
541	answers that it has no such door, and a box that has not been rolled yet simply
542	does nothing. Neither one turns it into a restart of everything.
543
544	### carlos db
545
546	Reads a copy of an instance's database. The platform keeps a replica of
547	every sleeping instance's database. This command restores that replica on
548	your machine. It never touches the live database.
549
550	```sh
551	carlos db pull --app hello --out hello.db
552	carlos db query --app hello --sql "SELECT count(*) FROM users"
553	```
554
555	`pull` saves the copy at the path you give with `--out`. The path must not
556	exist yet, and the file is readable only by you. `query` restores a fresh
557	copy, runs one query against it, and prints the rows to stdout with a header
558	row, separated by tabs. The copy is deleted when the query finishes.
559
560	Both say how recent the copy is: the last transaction it holds, and when
561	that transaction was written. A sleeping instance's copy is as recent as its
562	last sync before it went to sleep. A busy instance can be ahead of its copy.
563
564	To read the database as it was at an earlier time, add `--at` with a time
565	in RFC 3339 form, such as `--at 2026-10-07T14:00:00Z`. The copy holds every
566	change replicated before that time. It can be a little older than the time
567	you give, never newer, and the command says which transaction it stopped
568	at. How far back you can go depends on how much history the platform keeps
569	for the instance. Ask for a time earlier than that and the command names
570	the oldest time you can use instead.
571
572	The copy is read-only. A query that tries to change it fails with SQLite's
573	own error, `attempt to write a readonly database`. That is on purpose.
574	Changing a copy would change nothing on the instance, and a command that
575	seemed to succeed would say otherwise.
576
577	If the app has more than one instance, name one with `--instance`. Only instances that sleep have a database copy. An instance on your own fleet or in a pool cannot be read this way, and the command says which.
578
579	When an instance keeps its data in a different region from the console's, `carlos db query` runs in that region. A server there restores the copy, runs your query and sends back only the rows. `carlos db pull` fetches the copy's files from there and restores them on your machine. Both are encrypted for your terminal on the way, so nothing in between can read them, and nothing is printed or saved until all of it has arrived and been checked. In that region, a query's copy can be at most 2 GB once restored, the query runs for at most 5 minutes, and the rows can total at most 64 MB. A pull can fetch at most 2 GB.
580
581	### carlos run
582
583	Runs one command of your app against one instance, as that instance: the same build, user, directories, database and environment it serves with. Output streams back to your terminal about a second behind, and carlos exits with the command's own status.
584
585	```sh
586	carlos run --app shop --instance shop.example.com --file orders.csv -- import orders.csv
587	```
588
589	Any person who is a member of the account can run a command. A service credential cannot. The one exception is a `run` credential, which can run the commands it was minted with. [Runs from a service](#runs-from-a-service) says how.
590
591	Each --file is uploaded with the request and placed, under its own name, in the directory CARLOS_RUN_DIR names. Together they can come to 64 MB. The directory is deleted when the run ends.
592
593	The command runs for at most 30 minutes unless you pass --timeout, up to 6 hours. Ctrl-C cancels it on the box. A second Ctrl-C stops waiting, and the cancel still happens. If your terminal goes away the run carries on, and carlos run --app <app> --instance <host> --follow <id> picks its output up again. carlos prints that command when the run starts. Output is kept until carlos has read all of it, and for 24 hours at most.
594
595	A hibernating instance is woken first. A live one keeps serving, and the command opens the same database beside it. One run at a time per instance. carlos run needs the instance's box on the nested layout, and it is refused while the instance is moving.
596
597	carlos run exits with the command's own status, 128 plus the signal number when a signal ended it, 124 when it timed out, 125 when it was interrupted or the instance let go of it without a result, 130 when you cancelled it, 3 when the box refused it or did not pick it up within 60 seconds, and 2 when the console refused it.
598
599	carlos logs shows when each run started, who asked for it and how it ended. The command's output is not kept there. A service's run reads "started by service ci", with the person it named, if any, marked as the service's statement.
600
601	If you cannot tell whether a run started (the connection dropped as you sent it), start it with `--idempotency-key <key>`. The key is a string you choose: 1 to 128 characters of letters, digits, dots, underscores, colons and hyphens. Run the same command again with the same key and the same files, and carlos says `already started: run <id>` and follows that run, instead of starting a second. Each `--file` is hashed first, because a key only matches a request that is the same in every byte. The same key with a different command, timeout or file is refused with `idempotency_key_reused`. A key is yours alone: no other person or service can see it or collide with it. It is honoured for 24 hours, or until you have made 32 newer keyed admissions on that instance, whichever comes first. After that the same key starts a new run. If the run has since finished and its output was read, or has been cleared away, carlos says `run <id> was already started with this key, and its output is no longer kept` and exits 2: the run happened, but there is nothing left to follow.
602
603	Arguments are kept in the account's audit log, with who ran them. Never put a secret in an argument. Pass it in a --file, or keep it in the instance's environment.
604
605	#### Runs from a service
606
607	A server that has no person at a terminal, such as a back office that imports an account's data, can start and follow runs with a `run` credential. The account owner mints it:
608
609	```sh
610	carlos services create --role run --app shop --commands "import classic,import sync" fleet-runs
611	```
612
613	`--app` and `--commands` are both required. A `run` credential starts, follows, cancels, expires and acknowledges runs of its own app's instances, and does nothing else. No other role can run commands, `admin` included: a credential that can provision instances cannot run commands on them unless it was also minted for that.
614
615	`--commands` is a comma-separated list of prefixes, and each prefix is its words separated by spaces. A run is allowed when its arguments start with every word of at least one prefix, compared word for word and exactly. With only `import classic` granted, `import classic --slug acme` runs, and `import classical`, `import` and `import sync` are refused. **A prefix also allows every flag and argument after it**, so grant the shortest prefix that names the work you mean, and do not grant `import` when you mean `import classic`. A prefix of no words is refused. A word is valid UTF-8 of 1 to 128 bytes with no spaces, commas or control characters, and a credential may hold up to 32 prefixes. A refused command answers 403 `command_not_granted` before anything is stored.
616
617	The list is fixed at mint. `carlos services ls` shows it, rotating a credential keeps it, and there is no way to edit it: mint a new credential to change what it may run. People are not limited by any list.
618
619	A service sees and acts on only the runs its own credential started. Another service's run, a person's run and a run of an earlier credential that had the same name all answer `not_found`, as if there were no such run. Members of the account see every run, a service's included. When a service's run holds an instance, a service that did not start it is told that the instance has a run open and nothing more.
620
621	A service may say who a run is for, by sending `on_behalf_of` with an email address of at most 254 bytes. It is recorded on the service's word and checked for nothing but its shape, so it grants nothing. Every place a run is shown reads "run by service ci for ada@example.com (as stated by the service)", or "run by service ci (scheduled)" when it names no one. A person's run cannot send it.
622
623	#### The run API
624
625	`carlos run` is a client of these routes. A service calls them with its credential as a bearer token, and a person with a signed-in token. Under `/api/cli/apps/<account>/<app>/instances/<host>/runs`:
626
627	- `POST` creates a run. The body is multipart: a `meta` part first, as JSON (`argv`, `timeout_s`, `files` as a list of `name`, `size` and `sha256`, and `on_behalf_of`), then one `file` part per file, in the order and under the names `meta` gave. It answers `202` with the run's `id`, `host` and `state`.
628	- `GET /<id>?after=<seq>` pages the run's output from after chunk `seq`, and carries the result once the run has one.
629	- `POST /<id>/cancel` and `POST /<id>/expire` cancel a run, or give up on one no box has picked up.
630	- `POST /<id>/ack` with `{"last_seq": n}` says the output has been read, so the console can delete it.
631	- `GET ?key=<key>` answers `{"run_id": "..."}` for the run a key started, or `404`.
632
633	Every refusal a run route makes has a `code` beside its `error` sentence, for a caller that has no person to read the sentence. Not being signed in, a credential of the wrong role and an account the caller is not in are answered as they are on every other route, without a code, and tell themselves apart by status.
634
635	- `not_found`: no such instance, or no such run. For a service, also a run it did not start.
636	- `host_claim`: this console runs with host claims, which cannot yet run commands, so it refuses every run for everyone. The same refusal comes to a `run` credential.
637	- `rate_limited`: too many run requests. Wait a minute.
638	- `unavailable`: the console's store is not available. Try again.
639	- `record_failed`: the console could not record the run or the change. Try again.
640	- `read_failed`: the console could not read the run. Try again.
641	- `not_runnable`: the instance is of a kind `carlos run` does not work on.
642	- `moving`: the instance is moving. Run it once the move has finished.
643	- `run_open`: the instance already has a run open. A service that did not start it is not told whose it is.
644	- `malformed`: the request is not shaped as above.
645	- `invalid_request`: the command, timeout, files or sha256 values in `meta` are not acceptable.
646	- `files_too_large`: the files come to more than 64 MB.
647	- `import_uncommitted`: the instance is an import that is not committed yet.
648	- `command_not_granted`: this credential was not minted for that command.
649	- `on_behalf_of_invalid`: `on_behalf_of` is not an email address of at most 254 bytes, or a person sent it.
650	- `run_started`: the run has already started, so it cannot be expired.
651	- `run_ended`: the run has already ended, so it cannot be cancelled.
652	- `run_not_finished`: the run has not finished, so it cannot be acknowledged yet.
653	- `last_seq_mismatch`: the acknowledgement named a chunk other than the run's last.
654	- `idempotency_key_invalid`: the key is not 1 to 128 characters of letters, digits, dots, underscores, colons and hyphens.
655	- `idempotency_key_reused`: the key was used for a different request.
656	- `keys_not_ready`: this console does not take keys yet. See below.
657	- `file_hash_mismatch`: the bytes of a file are not the sha256 `meta` declared for it. This is a 400.
658
659	A create with an `Idempotency-Key` header is admitted at most once. A retry with the same key and the same request answers `202` with the same body and `"replayed": true`, and starts nothing. Keyed creates carry three rules:
660
661	- `meta` must declare each file's sha256, in either case. A file whose bytes do not match is refused as `file_hash_mismatch`.
662	- A refusal is not remembered, and a key is evaluated afresh on each try. A `run_open` that blocked the first try can succeed on the second, once the other run has finished.
663	- A 4xx on a retry means that attempt admitted nothing. An earlier attempt with the same key may have started the run, so confirm with `GET ?key=` before concluding that it never ran. A 5xx, a timeout or a dropped connection means unknown: retry with the same key.
664
665	A key is honoured for 24 hours, or until the same caller has made 32 newer keyed admissions on that instance, whichever comes first. The limit is per caller, so another caller can never evict your key. Do not retry a key past that: it starts a fresh run.
666
667	Keyed creates are off until the operator of the console turns them on, with `CARLOS_RUN_KEYS=on`, and answer `503 keys_not_ready` until then. The operator does that only once every box runs a build that keeps them. It is a one-way switch: a box rolled back to an older build afterwards forgets the keys silently, and a retry could then start a second run.
668
669	Deleting an instance deletes its keys with it. A retry that arrives after the host was deleted and created again starts a fresh run on the new instance.
670
671	### carlos schedule
672
673	Gives an app a timetable. A schedule is a time and a path: at each fire the
674	app's own instance gets a POST at that path, so the work is a route your app
675	already serves rather than a separate worker. The sub-verbs are `ls`, `set`,
676	`rm` and `run`.
677
678	```sh
679	carlos schedule set --app hello --name nightly --every 6h --path /jobs/nightly
680	```
681
682	`--every` takes whole minutes, from `1m` up to `720h`; `--cron` takes a
683	five-field expression instead, and you give one or the other, never both.
684	`carlos schedule ls` prints the declared schedules alongside what each
685	instance reports it will do next and what it did last. `carlos schedule run
686	--app hello --name nightly` asks for one out-of-band run on top of the normal
687	timetable, and every instance fires within about fifteen seconds.
688
689	The wording is deliberate: `set` records, `rm` removes, `run` requests. The
690	console writes one small object and every box serving the app notices on its
691	own within seconds; nothing is pushed at a box. The write itself wakes
692	nothing, and a hibernating instance is woken by its own runner when a tick
693	falls due. Schedules need a logged-in console; there is no direct-bucket
694	form.
695
696	### carlos rollback
697
698	Points a channel back at the version it was serving before.
699
700	```sh
701	carlos rollback --app hello stable
702	```
703
704	### carlos channels
705
706	What each of the app's channels is serving right now, with the tag if the
707	release earned one at stable.
708
709	```sh
710	carlos channels --app hello
711	```
712
713	### carlos channels resign
714
715	Re-signs a channel's current pointer with the sha256 of each build's manifest, so boxes that insist on digests accept it. It shows every digest first and signs only when you type yes. Use it on a channel nobody has promoted to since digests were turned on. It refuses until `carlos channels digests on` has been run for the account.
716
717	```sh
718	carlos channels resign --app hello stable
719	```
720
721	### carlos channels digests
722
723	Shows whether channel pointers carry manifest digests, and lists the channels that still do not. on and off are for platform operators: on tells upgraded writers to sign digests, and off stops them while a writer is being fixed. Turn it on only after every console, CLI and deploy script that promotes has been upgraded.
724
725	```sh
726	carlos channels digests
727	```
728
729	### carlos pipeline
730
731	Shows an app's release channels and the rules for moving a version through
732	them, and shapes that ladder. Bare `carlos pipeline` is the `show` verb: the
733	channels in order, plus any change still waiting on confirmations. `init`,
734	`add`, `set` and `remove` edit it.
735
736	```sh
737	carlos pipeline init --app hello --template edge-production
738	```
739
740	Two starter templates exist, `edge-production` and `full-ladder`. After
741	that, `--bake <dur>` holds a version for a while before the channel may adopt
742	it, and `--passkey`, `--promote-approvals N` and `--change-approvals N` set how
743	much human agreement a promotion into the channel, or an edit to the channel
744	itself, has to collect. `set` leaves any rule flag you did not type exactly
745	as it was, so two people shaping different rules do not overwrite each other.
746
747	A fresh app has one channel, `edge`, and prints as a single line rather than
748	a one-row table. Shaping a pipeline wants a logged-in console: there is no
749	bucket-direct editor for it. Promoting and rolling back are unaffected and
750	work in either mode.
751
752	### carlos releases
753
754	Every version shipped, newest first, with the label its author typed and the
755	tag it earned. Channels answer "what is running"; this answers "what is there
756	to run".
757
758	```sh
759	carlos releases --app hello
760	```
761
762	There are two sub-verbs. `carlos releases retention --keep 20` sets an ambient
763	prune policy, and `--dry-run` shows you what it would remove before you commit
764	to it. The safety list always wins over the number: channel pointers,
765	rollback history, tagged releases, anything inside the bake window, and each
766	box's own adopted versions are never pruned, so nothing promoted or recent
767	disappears by policy. `--off` goes back to keeping everything.
768
769	`carlos releases authorize <app> <version>` is for operators. It shows a build's manifest and file digests and, when you type yes, signs a promotion record for exactly those bytes. Every promotion writes one; this is for a build promoted before records existed, so an instance can be pinned to it or recovered onto it on a box that does not already have it.
770
771	### carlos version
772
773	`carlos version` prints the build id of the binary in front of you. It is the
774	first thing to check when a command behaves unlike this page describes.
775
776	`carlos version target` is a separate thing wearing the same word: it sets
777	the semver your ships count iterations under, so `carlos ship` can mint the
778	next one for you.
779
780	```sh
781	carlos version target --app hello 0.5.0
782	```
783
784	### carlos env
785
786	Plain per-app config vars: `set`, `unset`, `list`.
787
788	```sh
789	carlos env set --app hello LOG_LEVEL=debug
790	```
791
792	Values land in the app's default bundle. `--environment <name>` writes a named
793	bundle instead, layered on top of the default when config is materialized.
794	`carlos env environments --app hello` lists the bundle names an app has.
795
796	Binding a route to a bundle is a different verb depending on the route.
797	[carlos instances set-environment](/docs/cli#carlos-instances) is the one to
798	reach for: on a provisioned route the binding lives in the instance record,
799	and the reconciler converges the row from that record every pass — so
800	`carlos route --environment`, the box-local operator form, is reverted within
801	seconds and refuses such a row by name. `carlos instances create` takes
802	`--environment` too, which is how a route is born on the right bundle rather
803	than moved onto it.
804
805	Some routes are ones carlos proxies to without owning the process behind them.
806	Setting config for one of those writes the file and stops there. The program
807	carries on with the old values until its own supervisor restarts it. `set` and
808	`unset` say so, naming the routes it applies to, and say when they could not
809	work it out rather than staying quiet about it.
810
811	### carlos secrets
812
813	The same shape as `carlos env`, sealed. `list` prints key names and never
814	values.
815
816	```sh
817	carlos secrets set --app hello STRIPE_KEY=sk_live_example
818	```
819
820	Sealing uses a public key; the private half lives on the box that decrypts
821	them and never leaves it, in either mode. `carlos secrets genkey` mints the
822	pair, and it is always local to your terminal.
823
824	An empty value is refused rather than stored. A command substitution that
825	produced nothing looks exactly like a real value at the point you type it, and
826	a stored empty secret usually surfaces much later as a feature quietly not
827	working. Use `carlos env set` for a value that is meant to be empty.
828
829	### carlos instances
830
831	An app's instances are the places it actually runs. `carlos instances enable`
832	is the app's opt-in to provisioning, and until it is set the API refuses to
833	create anything. After that, `create`, `list`, `delete`, `set-upstreams`,
834	`set-channel` and `set-environment` work on records through the console, and
835	the box's reconcile pass turns a record into a live route. A password in
836	front of an instance's hostname is `carlos gate`.
837
838	An instance may sit at any free label under the account's platform domain,
839	but not at another app's platform apex. That is refused while the other app
840	is live and while it is in the Trash, since restoring it would bring the
841	clash back.
842
843	`--addr` on `create`, and `set-upstreams`, point an instance at TCP backends
844	of its own. Only a platform operator can set them, because the edge reaches
845	those addresses from inside the platform. If an instance already has
846	backends, you can still keep them, remove one, or change its other
847	settings; you cannot add a new one.
848
849	```sh
850	carlos instances enable --app hello
851	carlos instances list --app hello
852	```
853
854	`--health <slug>` narrows the listing to one state: `running`, `asleep`,
855	`not-responding` and the rest. That is usually what you want when something
856	is wrong.
857
858	The `DB` column is the size of each instance's SQLite database on disk,
859	including its write-ahead log, as its box last measured it. Boxes measure
860	every few minutes, and asleep instances are measured too. A dash means no box
861	has measured it: the instance is not running anywhere yet, it has not created
862	its database, or it keeps its data somewhere CARLOS does not manage.
863
864	`carlos instances list` shows two builds for each instance: the one installed
865	on its box, and the one serving its requests. They differ while a new build
866	rolls out, while an asleep instance waits for its next request, and after a
867	build has failed on that box. The REASON column says which.
868
869	`carlos instances retry` clears the failure hold on one instance, so the
870	build that failed there is tried again, as a canary:
871
872	```sh
873	carlos instances retry --app hello shop-1.hello.example.com
874	```
875
876	`--environment <e>` on `create`, and `set-environment` on a route that
877	already exists, bind one instance to one of the app's named config bundles.
878	This matters most for a route standing **alongside** a live one. Config is
879	per-app, so a second instance inherits the default bundle, and for anything
880	the app keys off a single origin — the `Origin` header it compares against,
881	magic-link URLs, a WebAuthn RP ID, the cookie `Secure` flag — that bundle
882	names the *other* host. The result is a host that serves every GET perfectly
883	and refuses every POST. Write the bundle with `carlos env --environment`,
884	bind it here, then restart the instance to pick the values up.
885
886	```sh
887	carlos instances set-environment --app jam --host new-queue-jam.bab.oncarlos.com --environment jam-canary
888	```
889
890	Without `--region`, a new instance starts in the app's region. An instance in a fleet starts where the fleet runs, and a pooled, static or replica instance starts in the home region.
891
892	An instance can also be an account's **replica**: a second instance of the
893	same app, holding a one-way mirror of the live instance's data, placed near
894	the buyers so a sale does not wait on the home region.
895	`carlos instances create --app jam --host checkout-uk.jam.bab.oncarlos.com
896	--region uk --role replica` provisions one. It needs `--region`: the replica
897	lives where you place it, and only the box carrying that placement mints it.
898	The live instance's own record names where replicas are wanted —
899	`--replicas ie,uk` on its create — and that is what publishes the placement
900	to the edge: the account's public hosts become servable in those regions,
901	with DNS latency routing still pointing buyers at the home. A replica never
902	holds the account's authoritative data, and until a batch of orders drains
903	home it holds order data in its own region — so the placement set is an
904	explicit decision on the home's record, never a default.
905
906	To start a new instance from a copy of another instance's database, add --from. The copy comes from the source's replica, so the source keeps serving. Both instances belong to the same app and account, and the copy stays in the source's region; run carlos move afterwards to put it somewhere else. The new instance gets its own environment, never the source's, and an empty private store when the app uses one. Add --at with a transaction id to copy an earlier point in the source's history.
907
908	```sh
909	carlos instances create --app shop --host canary.example.com --from www.example.com
910	carlos instances create --app shop --host canary.example.com --from www.example.com --at 4211
911	```
912
913	carlos instances import moves an instance the platform never ran onto the platform. Stop the old process, take a consistent snapshot of its database (VACUUM INTO works), and run the import with the address of the route it replaces. The console checks the file, the box swaps the route on its next pass, and from then until you commit, the host answers only requests that carry the verify token. With --verify, the CLI requests that path with the token and commits on a 2xx or aborts otherwise. Without it, the CLI prints the token and the two commands to finish by hand. Aborting puts the old route back exactly as it was.
914
915	```sh
916	carlos instances import --app shop --host main.shop.example.com --file main.db --replaces 127.0.0.1:8401 --verify /up
917	carlos instances import commit --app shop --host main.shop.example.com
918	carlos instances import abort --app shop --host main.shop.example.com
919	```
920
921	#### What an instance is started with
922
923	A hibernating instance is started by the box when a request arrives for it,
924	as `<your binary> --socket <path> --db <path>`: the unix socket to serve on,
925	and the SQLite database that is yours. On a box with the nested layout its
926	working directory is its own data directory, the one the database is in, so
927	a relative path such as `./data/blobs` lands beside the database and inside
928	the instance's namespace. (Under the flat layout it was `/`, so the same path
929	landed in the shared `/data`; `layout migrate` leaves such shared directories
930	where they are and names them in the agent log for an operator to place.)
931	Its environment is your app's config bundle, with the instance's own
932	`<host>.env` layered on top, plus a few variables the platform sets and that
933	no bundle can override:
934
935	| Variable | Value |
936	|---|---|
937	| `CARLOS_HOST` | the instance's hostname, exactly as in `carlos instances list`. Read this, never the socket's file name, which says nothing about the host on an isolated box |
938	| `CARLOS_REGION` | the box's region code, such as `ie` |
939	| `CARLOS_CONTROL_SOCKET` | the socket the platform listens on for your app's purge and scheduling calls |
940	| `CARLOS_PURGE` | `header` when the edge honours a `Carlos-Purge` response header |
941	| `CARLOS_ADMIN_TOKEN` | a secret shared with this one instance: the platform presents it when it calls your app, and your app presents it on `CARLOS_CONTROL_SOCKET` |
942
943	A seed (the `--seed` command of `carlos instances enable`) runs with the same
944	environment, `CARLOS_HOST` included, and no control socket.
945
946	A command started with carlos run gets the same environment, without CARLOS_CONTROL_SOCKET or CARLOS_PURGE, and three variables of its own:
947
948	| Variable | Value |
949	|---|---|
950	| `CARLOS_DB` | the instance's database, the path --db names when the instance starts |
951	| `CARLOS_RUN_ID` | this run's id |
952	| `CARLOS_RUN_DIR` | a directory holding the files passed with --file, under their own names. Always set, even with no files. Deleted after the run |
953
954	Its arguments are exactly what followed --, after your binary. The platform adds nothing. It runs in the instance's data directory. On a cancel, a timeout or an instance stop it gets SIGTERM, then SIGKILL 20 seconds later.
955
956	#### Storage of its own, for every instance
957
958	`--private-storage` on `enable` is one declaration that changes what every
959	later `create` does. The app says, once, that each of its instances gets a
960	config environment of its own with an object store bound to it — and from
961	then on `carlos instances create` derives the environment from the host's
962	first label, provisions that environment's bucket and IAM user, and delivers
963	the coordinates into the bundle the new instance reads. No `store create`, no
964	operator, no second command.
965
966	```sh
967	carlos instances enable --app jam --private-storage
968	carlos instances create --app jam --host acme.jam.bab.oncarlos.com
969	# created instance acme.jam.bab.oncarlos.com (channel stable, environment acme)
970	# private storage: bucket carlos-bab-jam-acme
971	```
972
973	This is the shape for a managed product with one instance per customer, and
974	the split is the point: declaring the policy is a signed-in person's act that
975	no service credential can perform, while spending it is the ordinary create
976	door an `instance`-role credential already reaches. So a public signup path
977	provisions a tenant's bucket without ever holding the power to create one.
978
979	Three things to know before you turn it on:
980
981	- The account needs an **approved storage entitlement** (`carlos store
982	apply`) and the deployment needs object storage configured. `enable`
983	refuses the policy rather than letting it fail once per signup.
984	- The host's first label becomes the environment name, so it must be a legal
985	one: lowercase letters, digits and hyphens, at most 32 characters, and not
986	`default`. A label that cannot name a store is refused at `create`, before
987	anything is written.
988	- **Nothing is ever torn down automatically.** Deleting an instance leaves
989	its bucket, exactly as it already leaves the parked database. `carlos store
990	status --app <a>` lists every store the app has, which is where an
991	abandoned one is visible; destroying it is the console's own action, behind
992	the typed bucket name and a passkey.
993
994	An app may have at most 1000 environment stores by default; a deployment sets
995	its own limit with `CARLOS_STORE_MAX_PER_APP`. The limit exists because each
996	store is an IAM user as well as a bucket, and AWS allows 5,000 IAM users per
997	account — so it bounds one app, not the deployment. Five apps at the default
998	would reach that quota between them.
999
1000	`--private-storage=false` turns the policy off. Instances created before it
1001	keep the bindings and buckets they already have.
1002
1003	**The edge-local door.** Some apps run their own processes on a box beside their instances: a router that serves many customers under one domain, a relay, a sync loop. Dialling each instance's socket directly stops working when an instance sleeps, moves to another box, or changes where its socket lives. The edge-local door is one socket those processes dial instead, wherever the instance is.
1004
1005	```sh
1006	carlos instances door-key create --app hello --label router
1007	curl --unix-socket /run/carlos/edge-local.sock \
1008	-H 'X-Carlos-Door-Key: cdk1.…' https://acme.hello.bab.oncarlos.com/
1009	```
1010
1011	The door handles a request the way the edge handles one from the internet. It wakes a sleeping instance, forwards to the box that holds a moved one, and applies the password gate. It only reaches the key's own app. Any other host answers as unknown.
1012
1013	The door believes no visitor address the caller sends. To your instance, a door request looks like one the box itself made to the public URL: it carries the box's public address, over `https`. If your app needs the visitor's address, have your process send it in a header your app signs and checks itself.
1014
1015	The door is off until a box sets `CARLOS_EDGE_DOOR=/run/carlos/edge-local.sock` and its agent restarts, and the calling process's user must be enrolled with `carlos bootstrap --socket-consumer <user>`. A revoked key stops working within 30 seconds, or within 10 minutes on a box that cannot reach the bucket. A key's record the console has not re-signed for 72 hours stops working too, so a console that stays down that long closes the door.
1016
1017	Typed on a box with no sub-verb, `carlos instances` does something different:
1018	it lists that box's own registry routes, backing and owning unit included.
1019	Off the box, [carlos routes](/docs/cli#carlos-routes) answers the same
1020	question through the console.
1021
1022	### carlos instances set
1023
1024	Sets how one instance takes new builds: auto or manual updates, the channel, a major version to stay on (stable only), or an exact build to pin. An empty value clears a setting, so the app's default applies again.
1025
1026	```sh
1027	carlos instances set a.shop.example.com --updates manual
1028	```
1029
1030	### carlos instances update
1031
1032	Asks the instance's box to install the newest build it considers ready, even with manual updates. The box chooses the build, and it runs on the next start. --fast-track skips the safety delay; only account owners can use it.
1033
1034	```sh
1035	carlos instances update a.shop.example.com
1036	```
1037
1038	### carlos sidecars
1039
1040	A sidecar is a member that nothing can dial: a process the platform runs for
1041	your app that binds no socket, has no hostname, and never appears in DNS or
1042	at the edge. It is for the work that has no request behind it — a bot that
1043	holds a connection out to somewhere else, a worker that wakes on a schedule,
1044	anything an inbound URL would be the wrong shape for.
1045
1046	`carlos sidecars enable` is the app's opt-in, and it is where the policy
1047	lives: `--args` is the argv the platform execs your release binary with, and
1048	it is app-wide rather than per-sidecar, because it is a property of what your
1049	current release can be asked to do. One binary runs every sidecar, and each
1050	child learns which one it is from `CARLOS_SIDECAR_NAME` in its environment.
1051
1052	```sh
1053	carlos sidecars enable --app eleven --args 'bot run'
1054	carlos sidecars create --app eleven bot-acme
1055	carlos sidecars ls --app eleven
1056	```
1057
1058	Sidecars hibernate by default, and a sleeping one has only two ways back up:
1059	its own schedule, or `carlos sidecars wake`. Nothing can dial it awake, which
1060	is the difference that matters when you are reasoning about one — an instance
1061	that goes to sleep is woken by its next request, and a sidecar has no next
1062	request. `--no-hibernate` on `enable` makes the app's sidecars always-on
1063	instead: the record exists, so the process runs.
1064
1065	```sh
1066	carlos sidecars wake --app eleven bot-acme
1067	carlos sidecars logs --app eleven bot-acme
1068	carlos sidecars rm --app eleven bot-acme
1069	```
1070
1071	`--idle` and `--max-run` on `enable` are the two ceilings. `--idle` (default
1072	5m) is how long a hibernating sidecar may sit with nothing to do before it is
1073	stopped; `--max-run` (default 1h) bounds a single run whether or not anything
1074	is happening, which is the guard against a worker wedged in a reconnect loop.
1075
1076	`--placement no-inbound` restricts an app's sidecars to boxes that have
1077	declared they serve no inbound traffic at all. The box declares it and the
1078	record asks for it, independently and on purpose: one is a fact about the
1079	network, the other a requirement of the work. Where they disagree the more
1080	restrictive reading wins.
1081
1082	Because a sidecar has no URL, there is no `X-Carlos-Version` header to read
1083	it back from, so `carlos sidecars ls` shows the build its box last started
1084	for each sidecar, beside the version its channel points at. One that has not
1085	started on any box reads as unknown rather than as current.
1086
1087	A console operator can pause sidecars for every account not named in
1088	`CARLOS_SIDECAR_ACCOUNTS`, a comma-separated list of account sqids. A paused
1089	account gets a 403 with the code `sidecars-paused` from `enable` and
1090	`create`. Its existing sidecars keep running, and `ls`, `wake` and `rm` still
1091	work. Unset, nothing is paused.
1092
1093	### carlos steering
1094
1095	Decides whether one instance's DNS answer varies by where the request comes
1096	from. `latency` opts the host into a Route53 latency record per armed edge,
1097	so the nearest one answers; `off` clears the opt-in and returns the host to
1098	the single static answer every requester shares.
1099
1100	```sh
1101	carlos steering --app hello --host hello.example.com latency
1102	```
1103
1104	The opt-in is recorded immediately and reaches the box's registry row at the
1105	next converge tick, but no DNS answer changes until the deployment itself
1106	has armed its Route53 steering converger. In v1 only instances on the shared
1107	pool can be steered; a pool-scoped route is refused, and the refusal names
1108	that as the reason.
1109
1110	A **static site on a platform hostname is steered from the moment it is
1111	created** — you do not have to ask. Static is the one kind every edge can
1112	serve on its own, straight from the channel pointer, so there is no reason
1113	for one edge to answer for all of them. `off` turns it back off if you want
1114	a single answer, and nothing re-applies the default to a host that already
1115	exists. Custom domains are not steered for you, because their DNS is yours:
1116	point them at the steering hostname the app's Settings page names.
1117
1118	### carlos routes
1119
1120	The app's routes on this deployment: where each one sends traffic, which
1121	channel it follows, which config environment it is bound to, and how often
1122	its database is replicated off the box. The environment column is the one
1123	worth knowing about, because it decides which sealed secrets the route's
1124	process is delivered. The replication column reads `default` until somebody
1125	sets a rung with `carlos route --replication`; the default syncs on every
1126	change.
1127
1128	```sh
1129	carlos routes --app hello
1130	```
1131
1132	It reads the console's own box, so on a multi-box fleet a short answer is not
1133	necessarily a complete one. The command says so every time it prints.
1134
1135	Its `DB` column is each route's SQLite database size, the same figure as in
1136	[carlos instances](/docs/cli#carlos-instances).
1137
1138	### carlos domains
1139
1140	Attaches a customer's own hostname to one of the app's routes, detaches it,
1141	or lists what is claimed.
1142
1143	```sh
1144	carlos domains attach --app hello www.example.com
1145	```
1146
1147	`--route` picks which route to point at, and you can leave it off when the app
1148	has exactly one instance record. The `list` verb joins the claims to the
1149	fleet's own readings sweep, so DNS state, certificate expiry and delegation
1150	are readable without leaving the terminal. The hostname comes live on the
1151	box owning its route at its next domains pass, certificate and all.
1152
1153	On a deployment that sets `CONSOLE_DOMAIN_PROOF=true`, a hostname is not
1154	served until you prove you control it. `attach` reserves the name and prints
1155	a TXT record to add, at `_carlos-challenge.<hostname>`. Once the record is
1156	there, run:
1157
1158	```sh
1159	carlos domains verify --app hello www.example.com
1160	```
1161
1162	If the record is already in place when you attach, `attach` verifies it
1163	straight away. Until then the name stays in the `unverified` state, and a
1164	reservation nobody proves lapses after seven days. Someone who does prove
1165	control can take it over sooner. Names under your own account's platform
1166	domain need no record. `--channel` on `verify` is only needed when the
1167	hostname will be the app's first route and the app follows more than one
1168	channel.
1169
1170	The console always refuses its own hostname, and names under another
1171	account's platform domain (`<their-account>.<apps-domain>`). By default it
1172	accepts every other name, including names under the console's domain and
1173	under the apps domain. That fits a self-hosted deployment, where one person
1174	owns the whole zone. A deployment that hosts apps for other people sets
1175	`CONSOLE_RESERVE_PLATFORM_DOMAINS=true` on the console. Then the console also
1176	refuses every name under the apps domain and under its own hostname, because
1177	those names belong to the operator and not to a tenant.
1178
1179	On a platform hostname, an app can set cookies for its own hostname or for
1180	its own account's platform domain (`<your-account>.<apps-domain>`), so your
1181	apps can share a cookie. Browsers would otherwise let an app set a cookie
1182	for the whole apps domain, and every other account's apps would get it. So
1183	when a cookie's `Domain` names anything wider, or another account, the edge
1184	removes the `Domain` and the cookie only goes back to the hostname that set
1185	it. Custom domains are left as they are.
1186
1187	#### One name per instance: `*.<parent>`
1188
1189	For an app with instances, attaching `*.<parent>` gives every instance a
1190	name under your own domain. An instance at `acme.bab.oncarlos.com` then also
1191	answers at `acme.example.com`. The label is the first part of the
1192	instance's own hostname.
1193
1194	```sh
1195	carlos domains attach --app teams example.com --route www.bab.oncarlos.com
1196	carlos domains attach --app teams '*.example.com'
1197	carlos domains list --app teams
1198	```
1199
1200	- **Create the instance, get the name.** A new instance gets its name with
1201	no further call, and a deleted one loses it.
1202	- **Prerequisites.** `<parent>` itself must already be attached to an app in
1203	the same account. Any app will do; a static site is common. A wildcard DNS
1204	record (`*`) for `<parent>` must point at this deployment.
1205	- **Who can attach it.** Only a signed-in person can attach or detach a
1206	catch-all, never the app's own token. Quote the star, or your shell
1207	expands it.
1208	- **What wins over it.** A hostname that has its own claim always wins over
1209	the catch-all, so `www.example.com` can still go to a different app.
1210	- **What it reserves.** While the catch-all is attached, no other account
1211	can claim the name of one of your instances. Publish the TXT record
1212	`_carlos-catchall.<parent>` = `carlos-catchall-account=<your account
1213	sqid>` and it reserves **every** `<label>.<parent>`, so a new team's name
1214	is safe before the team exists. `carlos domains list` shows which,
1215	`reserves:all` or `reserves:instances`, and names the record when it is
1216	missing.
1217	- **In `list`.** The catch-all shows as `per-instance` / `instances`, and
1218	each name the fleet is serving follows it, marked `via *.example.com`.
1219
1220	Each name gets its own certificate, as any attached hostname does. Let's
1221	Encrypt limits how many new certificates one domain gets in a week. An app
1222	expecting many new instances a week should also enable `--wildcard` on
1223	`<parent>`'s own claim, so one certificate covers every name.
1224
1225	```sh
1226	carlos domains detach --app teams '*.example.com'
1227	```
1228
1229	### carlos features
1230
1231	Sets or lists an app's own feature flags — switches the app defines, the
1232	platform stores and serves, and nobody but the app interprets.
1233
1234	```sh
1235	carlos features set --app titogo sandbox=on
1236	carlos features list --app titogo
1237	```
1238
1239	Keys and values are the app's vocabulary (`sandbox=on`, `beta-ui=v2`); the
1240	platform never reads them. `key=` with nothing after the equals clears a key.
1241	The app finds the map on the `deployment` block its instance token already
1242	fetches beside `managed_domains`, so a change lands on its next poll — no
1243	restart, no environment edit, no root-owned file on an edge box. Setting is
1244	an admin-role write and every change is audited under the app's Activity;
1245	the app's own instance credential can read the flags but never set them.
1246
1247	### carlos gate
1248
1249	Puts one shared password in front of an app's hostnames — for a static site
1250	or a prototype that should not be world-readable. Visitors see a plain page
1251	asking for the password and stay signed in for a week; the app behind it
1252	never sees the cookie.
1253
1254	```sh
1255	carlos gate set --app site
1256	carlos gate list --app site
1257	carlos gate clear --app site
1258	```
1259
1260	`set` reads the password from `--password` or, without it, one line from
1261	stdin, so it need not land in your shell history. Without `--host` the
1262	change applies to every instance of the app; name one to narrow it. Changing
1263	or clearing the password signs every visitor out at once. Custom domains
1264	attached to the app are gated along with the platform hostname. This is a
1265	curtain in front of the whole hostname, not accounts for the app's own
1266	users; an app with real sign-in keeps doing that itself.
1267
1268	### carlos logs
1269
1270	The app's own stdout and stderr, merged with the platform's events about it:
1271	wakes, restarts, and failures with the reason. A site served from files has
1272	no app stream, but its platform events still show up.
1273
1274	```sh
1275	carlos logs --app hello --since 1h -f
1276	```
1277
1278	`-f` follows, polling every two seconds until you interrupt it. `--grep` takes
1279	an RE2 pattern, `--stream app` drops the platform commentary, and `--host`
1280	narrows to one instance.
1281
1282	### carlos errors
1283
1284	One row per distinct error, newest first, with how many times it happened
1285	in the window and where. Two sources need no wiring at all: log lines the
1286	app tagged `error` (or `warn`, if you set the level) and 500s the app
1287	answered through the edge. Server-side and browser hooks come later. A 502,
1288	503 or 504 is not here: it says the instance could not answer, not that the
1289	request failed, and `carlos availability` has it.
1290
1291	```sh
1292	carlos errors --app hello --since 24h
1293	carlos errors --app hello --fp 3f2a9c1b5e7d0a4c
1294	carlos errors --app hello --set-level warn
1295	```
1296
1297	`--fp` expands one group to its occurrences. `--level warn` widens a single
1298	read without changing the setting; `--set-level` changes it for the app.
1299	The same view is the Errors tab on the app's dashboard.
1300
1301	### carlos availability
1302
1303	The requests an app's instances could not answer — every 502, 503 and 504
1304	the edge saw — grouped by cause rather than by page: the instance refused
1305	the connection (down, or mid-restart), closed it mid-request, timed out, no
1306	box of the fleet was connected, or the app answered 503 itself. One outage
1307	is one row however many pages it hit, and a scanner probing forty paths of
1308	a dead instance is one row too. The platform's own events about the
1309	instance — failed wakes and exits — are listed beside them.
1310
1311	```sh
1312	carlos availability --app hello --since 24h
1313	carlos availability --app hello --fp 3f2a9c1b5e7d0a4c
1314	```
1315
1316	`--fp` expands one cause to its occurrences, each with the path it was
1317	asking for. A request whose visitor left within five seconds is not
1318	counted; one who gave up after waiting longer is, as an outage.
1319	Records the edge filed before causes were recorded are grouped under
1320	"cause not recorded". The same view is the Availability tab beside Errors.
1321
1322	### carlos analytics
1323
1324	Page views, daily visitors, bytes served, top pages, top referers and named
1325	events, counted at the edge as it serves your app: no JavaScript, no cookie,
1326	no stored IP. Off until you turn it on, for one app or for the whole account.
1327
1328	```sh
1329	carlos analytics --app hello --set-count edge
1330	carlos analytics --app hello --range 7d
1331	```
1332
1333	`--range` is `today`, `7d` or `30d`; multi-day visitor totals are daily
1334	visitors summed. The same view is the Analytics tab on the app's dashboard,
1335	and the switch is on the Settings tab too.
1336
1337	When the deployment has a GeoIP database, page views are also counted by
1338	country. A visitor the database could not place counts as `unknown`. The
1339	Analytics tab credits the database the counts came from.
1340
1341	### Headers your app receives
1342
1343	On a deployment with a GeoIP database, every request the edge passes to your
1344	app carries the visitor's rough location:
1345
1346	| Header | Value |
1347	|---|---|
1348	| `X-Carlos-Geo-Country` | the country, as an ISO 3166-1 code such as `IE` |
1349	| `X-Carlos-Geo-Subdivision` | the state or province, as an ISO 3166-2 code such as `US-WA`. Only some databases have these |
1350	| `X-Carlos-Geo-Lat`, `X-Carlos-Geo-Lon` | a point, to two decimal places |
1351	| `X-Carlos-Geo-Source` | which database answered, such as `DBIP-City-Lite` |
1352
1353	Any of them can be missing. They are all missing when the deployment has no
1354	database, when the database has no answer for the address, and on calls
1355	through the edge-local door, where the caller is a server rather than a
1356	visitor. Treat a missing header as "unknown", never as an error.
1357
1358	The point is an area, not a position. These databases place an address to
1359	the right city at best, and often only to a region or a country.
1360
1361	The edge removes any `X-Carlos-Geo-` header a visitor sends, so the values
1362	your app sees are always the platform's. The visitor's address is still in
1363	`X-Forwarded-For` if you need it.
1364
1365	If you cache a page that differs by country, send `Vary:
1366	X-Carlos-Geo-Country`. The edge keeps at most eight variants of one URL, so a
1367	page with visitors from many countries will be served from cache less often.
1368	Don't vary on the coordinates: almost every visitor would get their own
1369	variant.
1370
1371	**Credit the database.** If your app shows visitors anything worked out from
1372	these headers, it must credit the database `X-Carlos-Geo-Source` names:
1373
1374	- `DBIP-…`: "IP Geolocation by DB-IP", linked to https://db-ip.com
1375	- `GeoLite2-…`: "This product includes GeoLite2 data created by MaxMind,
1376	available from https://www.maxmind.com"
1377
1378	### carlos geo
1379
1380	```sh
1381	carlos geo 81.2.69.142 --app hello
1382	carlos geo 81.2.69.142 --app hello --json
1383	```
1384
1385	The same answer the `X-Carlos-Geo-` headers carry, for an address that is
1386	not the current request: one from a log, a batch job or a webhook. It prints
1387	the country, subdivision, point and source, and the credit the database
1388	requires. An address the database can't place says so. Apps call the same
1389	door with their instance token:
1390	`GET /api/cli/apps/<account>/<app>/geo?ip=<address>`, at most 60 a minute.
1391
1392	### carlos store
1393
1394	Object storage for an app. An account owner applies once, and a deployment
1395	operator approves the account. After approval, `create` is the whole member
1396	workflow: it declares the store and provisions it when the deployment has an
1397	object-storage backend. Approval allows unlimited stores, with usage metered;
1398	the deployment's cloud account and its cloud provider's soft bucket quota remain
1399	the practical ceiling.
1400
1401	```sh
1402	carlos store apply --account bac --note "backups for the service"
1403	carlos store approvals
1404	carlos store approve --account bac
1405	carlos store create --app hello
1406	carlos store status --app hello
1407	```
1408
1409	`apply` is for an account owner. `approvals`, `approve` and `decline` are
1410	deployment-operator commands. `decline` requires a reason:
1411
1412	```sh
1413	carlos store decline --account bac --reason "storage is not available for this deployment"
1414	```
1415
1416	An owner may apply again after a decline. Declining an already approved account
1417	stops new automatic provisioning but does not tear down existing buckets or
1418	credentials. `carlos store grant` remains the per-app operator override for an
1419	unapproved account, for adopting an existing bucket with `--bucket`, and for
1420	repairing a store after approval is withdrawn.
1421
1422	`status` adds an account line when it has never applied, is pending or has been
1423	declined. An approved account needs no extra line. It then shows the app's
1424	declaration and provisioning state. A human
1425	account member may see the decline reason; the CLI does not print the
1426	application note. Tenant-facing responses withhold the deciding operator's
1427	address. Machine credentials receive state without the note or reason, while
1428	operator views can show the application details they need to decide.
1429
1430	If provisioning fails after `create` has declared the store, run `create` again
1431	to resume provisioning. A bucket name can remain unavailable for a short time
1432	after the cloud provider destroys it; during that destroy/re-create window,
1433	retry after the provider releases the name. `status` shows an incomplete store
1434	so you can tell this from an undeclared store.
1435
1436	Once the store is provisioned, `status` also shows how big the bucket is. That
1437	figure comes from a sweep the console runs once a day, so it carries the date
1438	it was measured and can be up to a day old. Sizing a bucket means listing
1439	every object in it, and `status` does not do that on each call. A fresh store
1440	reads "not measured yet" until the next sweep.
1441
1442	#### A store for one environment
1443
1444	A store normally belongs to the app: one bucket, one credential, delivered
1445	into the app's default config bundle, and every instance of the app reads it.
1446	`--environment` declares a store that belongs to one named config bundle
1447	instead — its own bucket, its own credential, delivered into that bundle, so
1448	only the instances bound to it can reach it. An instance gets a bucket of its
1449	own by being the only instance bound to its environment.
1450
1451	```sh
1452	carlos env set --app hello --environment acme APP_ORIGIN=https://acme.example.com
1453	carlos instances create --app hello --host acme.bab.oncarlos.com --channel edge --environment acme
1454	carlos domains attach --app hello --route acme.bab.oncarlos.com acme.example.com
1455	carlos store create --app hello --environment acme
1456	carlos store status --app hello --environment acme
1457	```
1458
1459	The instance sits at a free label under the account's platform domain; the
1460	customer's own hostname is attached to it afterwards. `--route` is required
1461	on `domains attach` once an app has more than one instance — the console
1462	refuses to guess which one a hostname belongs to.
1463
1464	`create`, `status`, `grant` and `rotate` all take `--environment`, and each
1465	acts on that store alone. Without it they act on the app's own store, exactly
1466	as they always have — nothing about an app with no environment stores changes.
1467
1468	`carlos store status --app hello` also names every environment this app has a
1469	store for. Tearing a store down is always an explicit act, so an environment
1470	nobody uses keeps its bucket and keeps billing until someone destroys it;
1471	this line is where you see one.
1472
1473	Two instances bound to the same environment share its store, the same way
1474	they share its config. That is visible rather than refused: `status` names
1475	the environment a store belongs to.
1476
1477	`carlos store rotate` mints a fresh credential and leaves the old one live
1478	until you run it again with `--finish`, so the app has a window to pick the
1479	new one up. Rotating, declaring and reading status are member verbs and want
1480	nothing but your bearer token. `grant` requires membership in the account and a
1481	deployment-operator address on `CARLOS_STORE_OPERATORS`; a non-member still
1482	receives the same 404 as everyone else. `approvals`, `approve` and `decline`
1483	require both the fleet-operator role and that allowlist entry, but the operator
1484	does not need membership in the account being reviewed.
1485
1486	#### carlos store scan
1487
1488	Virus scanning over the objects in an app's store. **Off by default and
1489	opt-in**: a deployment that sets nothing has no engine and runs no sweep. The
1490	verbs that need an engine — `grant`, `sweep`, `enforce on` — refuse with a
1491	sentence naming the variable that would wire one up. The rest keep working
1492	without one, deliberately; see below.
1493
1494	```sh
1495	carlos store scan request --app hello --plaintext
1496	carlos store scan status --app hello
1497	```
1498
1499	`request` is the member's verb, and it takes an attestation you cannot skip:
1500	either `--plaintext` or `--client-encrypted`. **A scanner cannot read
1501	ciphertext.** If your app encrypts objects before uploading them — which is
1502	the default posture across this family of platforms — then no engine can say
1503	anything about their contents, and a result claiming they are clean would be a
1504	true-looking sentence about bytes nobody examined. Attesting
1505	`--client-encrypted` records that honestly: the store is marked not
1506	applicable, no sweep runs, and `status` says so instead of showing coverage.
1507	There is no default answer because neither one is safe to guess.
1508
1509	The remaining verbs belong to a deployment operator on
1510	`CARLOS_STORE_OPERATORS`, the same list `carlos store grant` uses:
1511
1512	| verb | what it does |
1513	| --- | --- |
1514	| `grant --app <a>` | turn the engine on for this store |
1515	| `revoke --app <a>` | turn it off, and clear enforcement |
1516	| `sweep --app <a>` | run one budget of scanning now, instead of waiting for the hourly pass |
1517	| `enforce --app <a> on` | deny the app's own credentials read access to any object the engine tagged infected |
1518	| `enforce --app <a> off` | lift that denial |
1519
1520	`revoke` and `enforce off` work even when no engine is configured. That is
1521	deliberate: unsetting the engine on a deployment with an enforcing store would
1522	otherwise leave the denial in place with no way to clear it.
1523
1524	**What `status` will and will not claim.** The coverage line always names two
1525	numbers — how many objects were scanned, and how many were not. They are not
1526	complements. An object too large to stream past the engine was never judged at
1527	all, and a bucket full of them could report zero infections while nothing had
1528	actually been looked at. An archive that expands past the engine's limits is counted as not scanned, the same as an object too large to stream. A sweep still in progress says so and says which
1529	numbers are stale; a first sweep that has not finished claims no coverage at
1530	all. Signature-database age is printed, and called out when it is old, because
1531	a clean verdict from three-month-old definitions is a weaker claim than a
1532	fresh one. The engine's database version is reported as what it is **now** —
1533	a verdict is kept for as long as an object's bytes do not change, so a store
1534	that has been quiet for months can show full coverage made up entirely of older
1535	judgements. `status` prints how many verdicts this sweep actually reached and
1536	how many it carried forward, so the two are never confused.
1537
1538	**Enabled is not enforcing.** With scanning on but enforcement off, infected
1539	objects are recorded and still readable. `carlos store status` says which of
1540	the two you have.
1541
1542	The operator side is six environment variables on the console. Setting `CARLOS_CLAMD_ADDR` wires an engine. It does not switch scanning on. The switch is `carlos system antivirus enable`.
1543
1544	| variable | default | meaning |
1545	| --- | --- | --- |
1546	| `CARLOS_CLAMD_ADDR` | unset | where to reach `clamd`: `host:port`, or a socket path beginning `/`. Unset means no scanning at all. A [scanner host](https://github.com/carlosframework/platform/tree/main/infra/modules/scanner-host) module output gives you this value. |
1547	| `CARLOS_SCAN_TAG_KEY` | unset | required once an address is set, and the console refuses to boot without it. It signs every verdict, so keep it: without it, stored verdicts cannot be verified. |
1548	| `CARLOS_CLAMD_MAX_BYTES` | 25 MiB | objects larger than this are recorded as skipped without being fetched. Set it equal to `clamd`'s own `StreamMaxLength`. |
1549	| `CARLOS_CLAMD_MAX_SIG_AGE` | 168h | how old the signature database may get before `scan status` warns |
1550	| `CARLOS_STORE_SCAN_BUDGET` | 60s | how much scanning one app gets per hourly tick |
1551	| `CARLOS_STORE_SCAN_INTERVAL` | 24h | the floor before a *completed* sweep is run again. A sweep still in progress ignores it. That is how a bucket bigger than one budget ever finishes. |
1552
1553	`CARLOS_CLAMD_SOCKET` is still accepted as another name for `CARLOS_CLAMD_ADDR`. `CARLOS_STORE_SCANNER=clamav` is deprecated: it still wires the default socket `/run/clamav/clamd.ctl` and seeds the switch once, and the console logs a warning on every boot while it is set. `CARLOS_SCAN_TAG_KEY_PREVIOUS` is optional and holds the old key while you rotate.
1554
1555	The budget arithmetic is worth stating rather than discovering. An hourly tick
1556	at 60s is 24 minutes of scanning per app per day; at a few hundred objects a
1557	minute over `clamd`'s INSTREAM, a million-object bucket takes months, not days.
1558	The default is honest for small and medium stores. A deployment with large ones
1559	raises the budget or shortens the tick, and `scan status` says when a sweep has
1560	spanned more than one tick rather than implying its numbers are current.
1561
1562	Setting `CARLOS_CLAMD_ADDR` without `CARLOS_STORE_BACKEND` is refused at boot. A scanner with no object stores to scan is a misconfiguration, and one loud failure beats a pass that silently finds nothing forever.
1563
1564	Wiring an engine is not the same as switching scanning on. The deployment-wide
1565	switch is `carlos system antivirus` — see [carlos system](#carlos-system).
1566
1567	##### Request one object
1568
1569	`carlos store scan request` opts a store into scanning. To request a scan for one uploaded object, your app calls the scan API with an app-scoped `service` credential. The account owner creates that credential:
1570
1571	```sh
1572	carlos services create --account example --app hello --role service upload-scanner
1573	```
1574
1575	##### Requesting a scan
1576
1577	Send the credential's bearer token with both requests:
1578
1579	```http
1580	POST /api/cli/apps/{account}/{app}/store/scan/object
1581	{"key":"uploads/report.pdf"}
1582	```
1583
1584	```http
1585	GET /api/cli/apps/{account}/{app}/store/scan/object?key=uploads/report.pdf&wait=30s
1586	```
1587
1588	A separate worker checks for requests every 30 seconds, with a work limit per app. Busy queues can take longer. `carlos store scan status` reports bucket-sweep coverage; use the object endpoint for this upload's result. See [Receiving scan results](/docs/scan-callbacks).
1589
1590	##### Before scanning
1591
1592	The API scans the app's default store. It has no environment-store selector. The store must be provisioned, scanning must be enabled for the deployment, and a member's plaintext request must have been granted by an operator.
1593
1594	##### Callback keys
1595
1596	Polling works without callback keys. For callbacks, set `CARLOS_SCAN_WEBHOOK_KEY` to a newly generated 32-byte secret encoded as hex. The per-app callback secret delivered to your app is also hex-encoded; decode it before verifying signatures.
1597
1598	An operator replaces one app's callback secret with:
1599
1600	```sh
1601	carlos store scan rotate-webhook-key --app hello
1602	```
1603
1604	This changes only `hello`. A scan grant can also deliver a callback secret. Delivering changed config restarts affected apps.
1605
1606	### carlos relay
1607
1608	A STUN and TURN relay the platform runs for your app's WebRTC calls, so a
1609	browser can find its public address and, when two networks cannot reach
1610	each other directly, relay through the deployment instead of a third party.
1611
1612	```sh
1613	carlos relay enable --app meet
1614	carlos relay status --app meet
1615	```
1616
1617	`enable` turns it on for one app and delivers `CARLOS_RELAY` to the app's
1618	config: a base64-wrapped JSON list of regions, each with its ICE server URLs,
1619	a key and a username suffix. The app mints a TURN credential per call from
1620	the key (see the relay design spec, §2). `status` shows what was issued,
1621	which routes have picked it up, and the health of each region's relay,
1622	none of which needs box access. `disable` removes it; every allocation the
1623	app holds ends on the relays' next authority read, within a minute.
1624	`adopted --host <route>` is how a member tells the platform that a
1625	TCP-address route it restarted by hand now runs the new key — `status` names
1626	the routes waiting on that, since they are the one shape this platform
1627	cannot restart for you. `rotate --region <r>` is the operator's verb after
1628	staging a new master; `keygen` prints one. After a rotation, `status` names
1629	any master that is retiring and the time it may be dropped, box by box, since
1630	a region is done with a key only once every relay there has stopped loading
1631	it. `serve` is the unit's entry point and not a thing to run by hand.
1632
1633	`keygen` prints one line — a key id, a key, and `-` for "no retirement date"
1634	— for a region's master key. Stage that line in `/etc/carlos/relay-masters`
1635	on the region's boxes, mode 0640, root-owned, group `carlos`. The console
1636	needs the same key in a different shape: set its
1637	`CARLOS_RELAY_MASTERS_<REGION>` secret to `<kid>:<key>[,<kid>:<key>]`, the
1638	list form being what a rotation needs while two masters are live. The region
1639	name is uppercased with hyphens as underscores, since variable names refuse
1640	hyphens. The key is printed once and stored nowhere by this command. Adding
1641	a master to that secret issues nothing on its own — `rotate` is what moves
1642	the region onto it, and it refuses until every live relay there reports the
1643	new key id, so a rotation cannot mint credentials nothing can verify.
1644
1645	Know what that console copy exposes before you stage it. The console holds
1646	every region's master as sealed config so it can derive an app's keys.
1647	Sealed config is decryptable on every host, so a compromised host can read
1648	every region's master and, with a master, mint credentials for any app in
1649	that region, present and future, until the operator rotates. Per-region
1650	masters mean a relay box holds only its own region's, and rotating one
1651	region recovers that region.
1652
1653	Four console variables turn the relay on for a deployment.
1654	`CARLOS_RELAY_REGIONS` is a comma-separated list of the regions that run
1655	one, in the order an app hands them to the browser; unset means this
1656	deployment has no relay, and `enable` refuses with a sentence naming the
1657	variable. `CARLOS_RELAY_OPERATORS` is the addresses allowed to run
1658	`rotate`. `CARLOS_RELAY_MASTERS_<REGION>` carries the masters themselves as
1659	`<kid>:<key>[,<kid>:<key>]`, the region uppercased with hyphens as
1660	underscores. `CARLOS_PLATFORM_DOMAIN` is the domain under which
1661	`relay.<region>.` resolves; unset, it defaults to the apps domain with its
1662	first label dropped, which is an inference to verify against your own DNS
1663	rather than assume. The rest is infrastructure: the `A` record for
1664	`relay.<region>.<domain>` and the security group's `relay = true` are staged
1665	in the deployment's Terraform, region by region. Arm platform boxes only —
1666	a fleet box is the customer's own hardware, and a master staged there mints
1667	credentials for every app in the region.
1668
1669	`serve` is what `carlos-relay.service` execs. The unit only starts on a box
1670	that has a masters file, so a box with no key never listens; a box with one
1671	answers STUN to anyone and TURN to credentials an enabled app minted. It
1672	reads its configuration from `/etc/carlos/host.env`:
1673	`CARLOS_RELAY_PUBLIC_IP` (the address it advertises, when IMDS cannot say),
1674	`CARLOS_RELAY_DENY_CIDRS` (peer ranges it refuses, on top of the private
1675	ones it always refuses), `CARLOS_AWS_REGION`, `CARLOS_RELAY_MASTERS_PATH`,
1676	and the `CARLOS_RELAY_LIMIT_*` family.
1677
1678	### carlos email
1679
1680	Tenant email sending. `carlos email enable` is the whole path in one
1681	command: it declares the From address the app sends as, ensures a sending
1682	domain, waits for SES to verify it, and delivers SMTP credentials to the app
1683	as env vars under a prefix — `CARLOS_SMTP` unless `--env-prefix` names
1684	another.
1685
1686	```sh
1687	carlos email enable --app hello
1688	carlos email test --app hello --to you@example.com
1689	```
1690
1691	On a custom domain, `enable` prints the DNS records to publish and then
1692	keeps polling until SES confirms them, giving up after ten minutes unless
1693	`--timeout` says otherwise; a wait that gives up exits non-zero and names what
1694	SES never confirmed. `carlos email domains add` is that same half on its own,
1695	for a second domain on an app already declared.
1696
1697	A sending domain can be claimed by exactly one CARLOS account at a time. A
1698	claim that is never verified lapses after seven days, and then another
1699	account can add the domain. If the domain already has an SES identity that
1700	CARLOS did not create for your account, `domains add` refuses it and tells
1701	you which TXT record to publish (at `_carlos-mail.<domain>`) to prove you
1702	control the domain. Once that record is there, add the domain again. CARLOS
1703	sends with an identity adopted this way but never deletes it.
1704
1705	`carlos email domains rm --app hello example.com` is the way back out. A
1706	domain one app holds is unavailable to everybody else until something
1707	releases it. `rm` is that something: it stops the app sending from the domain and, when it was
1708	the last app using it, releases the claim so another account can add and verify
1709	the domain, and deletes the SES identity if CARLOS created it for you. It refuses while the domain is
1710	where the app's own From address sits — move that first with `carlos email
1711	enable --app hello --from <addr>`. A custom domain's DNS records are yours,
1712	so `rm` prints them rather than deleting them: the DKIM CNAMEs are dead with
1713	the identity, while the MX, SPF and DMARC records stay correct if you ever
1714	add the domain back.
1715
1716	`carlos email test` sends a real message and reports what SES said about it.
1717	It sends through a throwaway standalone credential it mints and revokes
1718	around the send, so it never needs the delivered credential's password, and
1719	it proves that mail leaves the building rather than that the console believes
1720	it should. The message is `multipart/alternative`, a plain-text part and an
1721	HTML part, which is the shape most app mail takes. Your app can send any MIME
1722	message it builds over the delivered SMTP credential: plain text, HTML,
1723	multipart, attachments. Nothing on the platform side reads or rewrites the
1724	body.
1725
1726	`status` shows every domain's per-region verification state, the day's count
1727	against the cap, and whether sending is paused.
1728
1729	`events` is what happened to each message: delivered, bounced, complained
1730	about or delayed, newest first, with what the receiving server said — the
1731	line that tells you whether a bounce was a mistyped address or a full
1732	mailbox. `--message` narrows it to one message by the id SES gave it when it
1733	was sent, `--to` to one recipient, and `--since` to a window. Events are kept
1734	for 30 days. Where SES names the credential that sent a message, its events
1735	belong to that app alone, so two apps sharing a sending domain do not see
1736	each other's recipients; where it does not, they go to every app on the
1737	domain, as the day's counts always have.
1738
1739	```sh
1740	carlos email events --app hello --since 24h
1741	carlos email events --app hello --to guest@example.com
1742	```
1743
1744	`events push` tells the app itself. Give it a path and the console POSTs each
1745	delivery, bounce, complaint, rejection and delay SES attributes to the app, as
1746	JSON, to that path on the app's own platform host
1747	(`https://<app>.<account>.<apps domain><path>`), retrying on a widening
1748	schedule for about a day when the app does not answer 2xx. Redirects are not
1749	followed. Only events SES attributed to the app's own sending credential are
1750	sent, so an app is never handed another app's recipients. An app behind
1751	`carlos gate` cannot receive them.
1752
1753	```sh
1754	carlos email events push --app hello --path /mail/events
1755	carlos restart --app hello
1756	carlos email events push --app hello
1757	```
1758
1759	Turning it on delivers `CARLOS_MAIL_EVENTS_KEY` into the app's config — an
1760	Ed25519 public key, standard base64 — which the app reads after a restart.
1761	Every delivery carries a `Carlos-Signature: t=<unix seconds>,ed25519=<base64>`
1762	header over the bytes `carlos-mail-event`, a newline, the `t` value, a newline,
1763	and the request body exactly as received. An app should refuse a signature
1764	that does not verify, a `t` more than five minutes from its own clock, and a
1765	body whose `account` and `app` do not name the host the request arrived on —
1766	`<app>.<account>.` — since the key is the deployment's, not the app's. The
1767	body's `id` is the same on every retry of one event. Bare, `events push` shows
1768	where events go and how delivery has been going; `--off` stops it and takes the
1769	key back out. While an app is in the trash, what was queued for it is given
1770	up on and new events are dropped, not held for a restore. An app that accepts
1771	nothing for a day — every attempt answered with anything but 2xx — has its push
1772	suspended: what is waiting is given up on, nothing new is queued, and `events
1773	push` says so. Running `events push --path` again turns it back on.
1774
1775	`credentials create` mints a standalone SMTP credential for something not running on CARLOS — a laptop, a
1776	cron box — printed once on that command's output and nowhere else. `rotate`
1777	mints a fresh delivered credential and leaves the old one live until you run
1778	it again with `--finish`. `pause` and `resume` are operator verbs. None of it
1779	has a direct-bucket form; all of it wants a logged-in console.
1780
1781	### carlos ledger
1782
1783	Append-only hash-chained ledgers an app can publish. `append` adds one JSON
1784	entry to a chain, `head` prints that chain's current head, `list` shows the
1785	app's ledgers and what it publishes, and `publish` decides which of them are
1786	served publicly.
1787
1788	```sh
1789	carlos ledger append --app hello carbon entry.json
1790	```
1791
1792	`blob` uploads files to a chain as content-addressed blobs and prints each
1793	one's sha, so an entry can point at it. Uploads are create-only and capped at
1794	4 MB a file, and re-uploading identical bytes does nothing.
1795
1796	Ledgers have no direct-bucket mode at all: every sub-verb except `verify`
1797	goes through the console, because a contributor's machine is never given
1798	bucket credentials. `carlos ledger verify` walks a published chain over plain
1799	HTTPS and re-hashes every entry. It needs no credentials at all: anybody can
1800	check a ledger you publish, including you, from a machine that has never been
1801	logged in.
1802
1803	### carlos vet
1804
1805	Checks a shipped release against the platform contract before anyone promotes
1806	it. The manifest has to exist and parse, and every artifact's stored bytes
1807	have to match the sha256 and size it claims.
1808
1809	```sh
1810	carlos vet --app hello --version a1b2c3d --boot
1811	```
1812
1813	`--boot` goes further and runs the binary: it must accept `-socket <path>`,
1814	serve HTTP on that socket, and answer `GET /healthz` with a 200 within ten
1815	seconds. That is the entire app contract, and this is the cheapest place to
1816	find out you have broken it.
1817
1818	### carlos update
1819
1820	Replaces the `carlos` binary on your workstation with the latest published
1821	release, checking the signed checksums before it swaps anything. Where a
1822	package manager owns the install, it prints the `brew` or `apt` command
1823	instead of fighting it.
1824
1825	```sh
1826	carlos update
1827	```
1828
1829	It refuses to run as root, and refuses on a box: box binaries are rolled by
1830	the platform, and a self-updating box would fight that machinery. `-y` skips
1831	the confirmation, though it still wants a terminal.
1832
1833	### carlos skills
1834
1835	The platform publishes its agent skills at
1836	`/.well-known/agent-skills/index.json`, per the [Agent Skills Discovery
1837	draft](https://github.com/cloudflare/agent-skills-discovery-rfc). `carlos
1838	skills` lists what is published; `carlos skills <name>` fetches one, checks
1839	its sha256 digest against what the index claims, and prints the SKILL.md to
1840	stdout — a mismatch means corruption or tampering, and the command refuses
1841	to print it.
1842
1843	```sh
1844	carlos skills
1845	carlos skills getting-started > SKILL.md
1846	```
1847
1848	`-index <url>` points it at another deployment's index, for a self-hosted
1849	site or a test.
1850
1851	## Your deployment
1852
1853	The verbs for the person who owns the boxes. Some of these run against a
1854	console like the app commands above; others open a box's registry directly
1855	and only make sense while you are standing on it.
1856
1857	### carlos bootstrap
1858
1859	Prepares a host to be a CARLOS box: the service user, the data directories,
1860	the systemd units. Run it once per host, as root.
1861
1862	```sh
1863	carlos bootstrap
1864	```
1865
1866	`--root <dir>` writes the files into a staging directory instead, creating no
1867	users and running no systemctl, so you can read what it would do before it
1868	does it. `--offcloud` is for a box outside EC2, where credentials come from a
1869	staged 0600 file instead of an instance profile.
1870
1871	### carlos accounts
1872
1873	Accounts are the tenancy primitive: apps, fleets, credentials and bucket
1874	prefixes all hang off one. `create` mints an account and prints the sqid
1875	every object underneath it is keyed by.
1876
1877	```sh
1878	carlos accounts create --name acme --owner someone@example.com
1879	```
1880
1881	`--owner` is the path for a box or a CI job, where there is no browser to log
1882	in with. It needs no identity of its own and takes precedence over any
1883	logged-in one.
1884
1885	`carlos accounts migrate` copies an app's objects to an account-qualified
1886	prefix and re-stamps its routes. Run it with `--dry-run` first; it prints the
1887	plan and touches nothing.
1888
1889	### carlos accounts plan
1890
1891	Shows or changes an account's plan. It is a deployment-operator command, and it only works where the console has a plan catalog (see [Plans](plans)).
1892
1893	```sh
1894	carlos accounts plan --account bac
1895	carlos accounts plan --account bac --set personal --pack apps=2 --reason "needs more apps"
1896	carlos accounts plan --account bac --set free --sponsored limits.json --reason "existing account, sponsored"
1897	```
1898
1899	Without `--set` it prints the plan, who set it and why, and every limit the account has. With `--set` it changes the plan and prints the result. `--reason` is required for a change. `--pack` can be repeated. `--sponsored` names a JSON file of limits; each one replaces the plan's limit where it is more generous.
1900
1901	### carlos fleets
1902
1903	A fleet is a group of remote boxes an account owns, dialing in to this
1904	console over a channel named `<fleet>/<label>`. Create one against the
1905	customer's own data bucket, then register each box.
1906
1907	```sh
1908	carlos fleets create --bucket acme-data acme
1909	carlos fleets add-box acme pi-1
1910	```
1911
1912	The box's bearer token prints exactly once, at `add-box`. The console keeps
1913	only its hash, so losing that output means `rotate-token`; a second `add-box`
1914	will only refuse the label. Put it straight into the box's credential store.
1915
1916	Rotation is not revocation. A fresh token refuses the box's next dial-in with
1917	the old one, but it does not evict a channel the box is already holding. If a
1918	credential may be compromised, `detach-box` is the command that actually
1919	stops it.
1920
1921	### carlos services
1922
1923	Credentials a server holds rather than a person: a CI job that ships, a
1924	sidecar that reads instances, an app that mints an instance per customer at
1925	signup. Reach is fixed at mint, so adding you to another account later does
1926	not widen it, and it has its own rate-limit budget.
1927
1928	```sh
1929	carlos services create --role deploy --app hello --rungs canary,edge ci-shipper
1930	```
1931
1932	Pick the narrowest role that works. `publish` ships releases and manages the app; `instance`
1933	provisions an app's instances — create, delete, upstreams and moves, plus its
1934	domains and logs — which is the role a self-serve signup path wants; `operate`
1935	administers; and `admin` includes `store grant`, which mints a path-scoped IAM
1936	user, so treat that one as a real handover. `--app` binds the credential to
1937	one app, which also means it cannot create apps.
1938
1939	`deploy` is for a CI pipeline: it ships, and promotes or rolls back only
1940	the rungs named with `--rungs` (`canary`, `edge`, `beta`; never `stable`).
1941	It must name one app and cannot hotfix. If the app has a pipeline, it may
1942	move `canary/<name>` channels only. CI reads it from
1943	`CARLOS_SERVICE_TOKEN` with `CARLOS_CONSOLE`.
1944
1945	`run` is for a server that starts commands of an app against its instances, and no other role does that. It is minted with `--app` and `--commands`, and it can run only the commands it names. See [Runs from a service](#runs-from-a-service); it also says what a prefix allows.
1946
1947	One door stays shut to every role: `carlos instances enable`, which writes the
1948	manifest bounding which domain, pool and regions an instance may be created
1949	in — and, with `--private-storage`, whether each instance gets an object store
1950	of its own. A person enables the app once and the credential provisions inside
1951	those bounds afterwards, including minting a tenant's bucket it could never
1952	have created directly.
1953
1954	A manifest's `regions` allow-list compares the home as `""`, so listing the
1955	home's code does not admit the home.
1956
1957	The secret prints once, same as a fleet token, and the same distinction
1958	applies at the other end: `rotate` refuses the credential's next call with
1959	the old secret, `revoke` stops it whatever secret it holds.
1960
1961	### carlos add
1962
1963	Writes a route straight into a box's registry: this hostname, served by this
1964	app, following this channel. It is the box-local, operator-side counterpart
1965	to an instance record.
1966
1967	```sh
1968	carlos add --app hello --socket /run/hello.sock hello.example.com
1969	```
1970
1971	`--channel` sets what the route follows, `--kind` picks instance, service or
1972	static, and `--addr` takes a TCP upstream where the app is not a socket
1973	tenant. `--unit console.service` names the systemd unit that owns the route's
1974	process, which is what lets adoption cycle it; leave it off for an ordinary
1975	exec child.
1976
1977	### carlos remove
1978
1979	Drops one of this box's routes. It stops routing the hostname and never
1980	touches the instance's database.
1981
1982	```sh
1983	carlos remove hello.example.com
1984	```
1985
1986	### carlos route
1987
1988	Changes one thing about a route that already exists. Each axis is its own
1989	operation and the command refuses to combine them, because a capability grant
1990	and a repoint are different acts with different safety rails, and a command
1991	that did both would commit them together.
1992
1993	```sh
1994	carlos route --host hello.example.com --channel edge
1995	```
1996
1997	`--channel` and `--environment` repoint, and those two may be given together.
1998	`--grant` and `--revoke` change a capability. `--hibernate` lets the route sleep
1999	when idle. `--backing` and `--unit` are two answers to one question — who owns
2000	this route's process — so the command will not let you write both.
2001
2002	`--replication` picks how often the route's database is copied off the box:
2003	`stream` syncs on every change, about once a second; `batched` at most every
2004	five minutes; `daily` at wake and then once a day. The rung you pick is a
2005	trade, and it is the loss window you are trading. `daily` on a box that dies
2006	loses up to a day of writes. What it buys is real money on a large database
2007	that is written rarely — a 600 MB file compacted a few times a day costs the
2008	same in traffic whether or not anything changed, and that traffic was 97% of
2009	one day's replica writes across a whole fleet before anyone could turn it
2010	down.
2011
2012	```sh
2013	carlos route --host hello.example.com --replication batched
2014	```
2015
2016	`--replication -` puts the route back on the default, which behaves exactly
2017	like `stream`. It only applies to routes that hibernate and whose row names a
2018	database: those are the ones the activator replicates, and everything else on
2019	the box is covered by the box's own replicator at its own interval. The
2020	command refuses both cases rather than writing a setting nothing would read —
2021	a route that does not hibernate, and one that hibernates with no `--db` path
2022	on it, which the box replicator cannot tell apart from any other file and so
2023	sweeps up anyway. A change takes effect the next time the instance wakes.
2024
2025	The rung lives only as long as the sleeping does. Stopping a route from
2026	hibernating clears it — with `carlos route --hibernate=false`, and equally
2027	when an instance record turns hibernation off and the box converges — because
2028	from that moment the box replicator owns the database and the rung would name
2029	an interval nothing honours. The command says so when it clears one. Turning
2030	hibernation back on starts from the default again.
2031
2032	Repointing the channel is the half to be careful with. Adoption converges a
2033	route onto whatever its channel currently points at, so moving the channel
2034	changes what production serves on the next pass; the command resolves the
2035	target pointer first and refuses a version change unless you pass
2036	`--allow-version-change`; `--dry-run` runs the same rails and prints the plan
2037	without writing. And once a route is `Provisioned`, its channel and
2038	environment belong to its instance record: `carlos route` refuses both axes
2039	on such a row and names the record, because the reconciler would revert you
2040	within seconds anyway.
2041
2042	### carlos release-keygen
2043
2044	Mints the deployment's root signing key, or a scoped key that can sign only
2045	certain rungs of the ladder.
2046
2047	```sh
2048	carlos release-keygen --scope canary,edge --name builder
2049	```
2050
2051	With no flags it prints a fresh root pair: the key goes into your secret
2052	manager, the public half into host config. With `--scope` it prints a key and
2053	a grant. The grant carries no secret material but has to travel with the key,
2054	because a scoped key on its own signs pointers nothing will accept. Only the
2055	root key can mint a grant, which is what stops a scoped key widening itself.
2056
2057	### carlos proof
2058
2059	Makes and carries the proofs a box can require before it adopts a build. A
2060	proof is signed by a proving deployment (dev, usually) and says it ran these
2061	exact bytes healthily. A box whose proof policy lists an app adopts a new
2062	build of it only with a proof, or when it has run those bytes well itself.
2063
2064	```sh
2065	carlos proof keygen
2066	```
2067
2068	`keygen` runs as root on the proving box. It writes a fresh signing key to
2069	`/etc/carlos/proving.key`, readable by root only (`--out` puts it somewhere
2070	else), and prints the public half. That public half goes into the `require`
2071	entry of each requiring box's `/etc/carlos/proof-policy.json`. It never
2072	overwrites a key that is already there.
2073
2074	```sh
2075	carlos proof get --app console --sha256 <sha256> > proof.json
2076	```
2077
2078	`get` reads a build's proof from the proving deployment's bucket and prints
2079	it. `--sha256` is the build's artifact hash. `--prover` picks whose proof,
2080	when there is more than one. When there is no proof yet, it says how long the
2081	build has been live on the proving box, if it has been seen there.
2082
2083	```sh
2084	carlos proof put --app console proof.json
2085	```
2086
2087	`put` writes that proof into the requiring deployment's bucket, where its
2088	boxes look for it. It checks the proof's shape but not its signature. The
2089	requiring box checks that against the key in its policy. Putting the same
2090	proof twice is fine. A different proof for the same build and prover is
2091	refused.
2092
2093	`get` and `put` work only with `CARLOS_DEPLOYMENT_BUCKET` or
2094	`CARLOS_DEPLOYMENT_DIR` set, because a proof lives only in a bucket.
2095
2096	#### carlos proof history
2097
2098	Read or reconstruct a box's root release history. These operator commands use
2099	`CARLOS_DEPLOYMENT_BUCKET` (or `CARLOS_DEPLOYMENT_DIR` for a local store),
2100	`CARLOS_RELEASE_PUBKEY` and the deployment root signing key in
2101	`CARLOS_RELEASE_KEY`. Scoped signing keys cannot authorize this work.
2102
2103	```sh
2104	carlos proof history status --bucket <bucket> --box <box> --enrollment <id> --account <sqid> --app <app> --host <host> --state status.json
2105	```
2106
2107	The command saves its signed request before delivery, then waits up to 30
2108	seconds for a signed response from the enrolled box. The JSON response includes
2109	category completeness, revisions, pending reconstruction and retained recovery
2110	outcomes. A successful status read does not mean migration is complete.
2111
2112	To reconstruct history, prepare a JSON `HistoryReconstruction` decision using
2113	the identity, revision, app revision, bake revision and host sources from that
2114	response. Set `schema` to `1`, `kind` to `carlos-release-history`, a unique
2115	`operation` and the current `issued_at` time. Name every affected host and its
2116	revision and refusals. Explicitly state the historical `production`,
2117	`production_set_at` and `holds` bounds; hold windows use nanoseconds. Review
2118	those decisions before signing. For an interrupted reconstruction, `replaces`
2119	must name the exact `pending_digest`. Root retains the pending decision's denies.
2120
2121	```sh
2122	carlos proof history reconstruct --bucket <bucket> --box <box> --enrollment <id> --account <sqid> --app <app> --host <host> --input decision.json --state reconstruction.json
2123	carlos proof history resume --bucket <bucket> --state reconstruction.json
2124	```
2125
2126	`reconstruct` succeeds only after the box confirms the exact signed decision.
2127	It does not retire independent recovery outcomes or establish a fresh install.
2128	`retire` takes the same flags with a `HistoryRecoveryRetirement` decision:
2129	`schema: 1`, `kind: carlos-release-recovery-retirement`, `identity`, `operation`,
2130	`issued_at`, `binding`, `source`, `attempt_id`, `outcome_digest`, `verified_id`
2131	and `verified_digest`. The witness must be a later root-verified success.
2132	Pending originals and unrelated outcomes remain restrictive.
2133
2134	Use a new `--state` file for each decision. Keep it if the command times out or
2135	is interrupted. `resume` reuses the saved authorization and challenge without
2136	a signing key. It does not refresh their 24-hour validity. `--wait` changes the
2137	readback wait. A delivery acknowledgement alone never reports completion.
2138
2139	### carlos economics
2140
2141	Records one month's AWS bill so the console's economics dashboard has a real
2142	number to divide by.
2143
2144	```sh
2145	carlos economics bill --month 2026-08 --total 412.55 --line AmazonEC2=55.00
2146	```
2147
2148	Operator only, and the console answers everyone else with the same bare 404
2149	it gives an unknown route. A "not found" here almost always means your
2150	account lacks the fleet-operator bit, not that you typed the path wrong.
2151
2152	`carlos economics backhaul` shows what serving through region edges costs
2153	the platform this period: the part of each account's egress that was
2154	backhauled from its home box, priced at the rate card's inter-region
2155	$/GB and absorbed — it appears in nobody's bill.
2156
2157	```sh
2158	carlos economics backhaul --period mtd
2159	```
2160
2161	### carlos status
2162
2163	`--live` prints what every box in the fleet last reported about itself:
2164	release, tunnel state, memory, load, free disk, and whether its own alerts
2165	have anywhere to go. A box that has stopped reporting shows as UNREACHABLE
2166	with how long it has been dark, and none of its stale readings.
2167
2168	The last column reads `alerts=sns`, `alerts=webhook` or `alerts=webhook+sns`
2169	when a box can deliver its alerts, and `alerts=not-delivered` when it has no
2170	sink configured. `alerts=sns-unavailable` means a topic is set but the box
2171	could not build its SNS client, so nothing is delivered through it. A dash
2172	means the box runs an older binary and did not say.
2173
2174	```sh
2175	carlos status --live
2176	```
2177
2178	`--instances` answers the other question an operator has about a fleet: who is
2179	using what, right now. One row per awake instance across every box, with the
2180	account and app it belongs to and what it is consuming.
2181
2182	```sh
2183	carlos status --instances --account bac
2184	```
2185
2186	It goes through the console rather than the status bucket, so it needs no AWS
2187	credentials — which is the point of it existing. Operator only: a token
2188	without the fleet-operator bit gets the same bare 404 an unknown route gets,
2189	so it learns neither the rows nor whether it holds the bit. The path itself is
2190	not a secret — a wrong-method request gets a 405 from the router, like every
2191	other route here — but the answer is, because it names other people's
2192	accounts.
2193
2194	Two things it cannot see, both printed under the table. A sleeping instance
2195	has no process and so no reading, so it does not appear. And the readings are
2196	fleet-wide while the account each one is attributed to comes from the
2197	console's own box's route registry, so on a multi-box deployment an instance
2198	running elsewhere can show up under `(platform)` rather than under its tenant.
2199
2200	`--layout` shows which layout each instance is on, box by box. `flat` is the
2201	shared `/data` every instance used to live in. `nested` is the instance's own
2202	directory, where it cannot see its neighbours. `fenced` means a move between
2203	the two is in progress or stopped part way, and `conflict` means two routes
2204	claim the same directory; neither wakes until that is resolved. `foreign`
2205	rows have paths the layout pass does not recognise and leaves alone, and
2206	`external` rows are run by another supervisor. Like `--instances`, it goes
2207	through the console and is operator only.
2208
2209	A box that runs `nested` instances beside `flat`, `foreign` or `external`
2210	ones gets a warning after the table. Each of those still shares the agent's
2211	user and view of `/data`, so the nested ones on that box are not isolated
2212	until none remains. An instance that is awake when the layout changes is left
2213	running for up to an hour so it can move while asleep; after that it is
2214	stopped briefly and moved, and wakes nested on its next request. The `REASON`
2215	column says how long it has left. Anything found in an instance's new
2216	directory that the move did not put there is set aside under
2217	`/data/layout/aside/<host>/` and the move goes on; the `REASON` column notes
2218	what was set aside and when. An instance that has been stopped for its move
2219	and then cannot be moved (a file of its has a second name, or its directory
2220	is a symlink) stays stopped and shows as `fenced` with the reason, across
2221	agent restarts, until the reason is cleared or the layout is reverted; it
2222	never goes back to running flat on its own.
2223
2224	```sh
2225	carlos status --layout
2226	```
2227
2228	`--dry-run <dir>` is the self-hoster's leak check. It builds a throwaway fleet
2229	stuffed with fixture secrets, runs it through the real publish path into a
2230	scratch directory, reads every published object back, and fails if any
2231	fixture escaped. Run it before you ever point this binary at a real bucket.
2232	Neither mode publishes anything: the public status page is written by each
2233	box's own agent tick.
2234
2235	### carlos system
2236
2237	Two things that are true of the deployment rather than of one app: the
2238	platform's own release channel, and the deployment-wide settings an operator
2239	owns.
2240
2241	#### The platform's own binary
2242
2243	`update` promotes a public release (or, with `--file`, a binary you built) to
2244	`canary` or `stable`; every box on that channel fetches it through its normal
2245	store, verifies it against the signed pointer, installs it, restarts its
2246	instance-control service and then its edge, and rolls back to the previous
2247	binary if the new one does not come up and stay up. `status` shows each box's platform, channel, running build, the channel's
2248	target, and the last apply outcome with its reason. `rollback` points a channel
2249	back at the release before this one, and every box on it rolls back the same
2250	way it rolled forward. All three talk to your console. You need to be an owner
2251	of the ops account, or on the console's `CARLOS_SYSTEM_OPERATORS` list. That
2252	list is for people who roll releases without owning the account. It covers
2253	`status`, `update` from a public release, and `rollback`, but not `--file`: the
2254	console signs an uploaded build with the deployment key and every box runs it,
2255	so only an owner can ship one.
2256
2257	A box that has isolated instances refuses to install a platform release from before the instance layout, whether by update or rollback, until its layout has been reverted and every instance reports `flat`. The box judges the candidate by the units it ships and, when the candidate's version is a release number, by that number against the first release on which isolated instances run; a locally built binary named by a commit is judged by its units alone. The console refuses the same move before it reaches a box: `update` to a named release from before the layout, or `rollback` onto one, is refused while any box reports an instance that is `nested`, `fenced` or `conflict`, or has not reported its layout at all.
2258
2259	```sh
2260	carlos system update --channel canary
2261	carlos system status --channel canary
2262	carlos system update --channel stable --spread 10m
2263	carlos system rollback --channel stable
2264	```
2265
2266	#### Cancel an unpublished activation
2267
2268	If an unpublished platform update holds its channel, an ops account owner or
2269	system operator can cancel that reservation:
2270
2271	```sh
2272	carlos system cancel-activation --channel stable --activation ACTIVATION_ID
2273	```
2274
2275	Use the activation ID from the refused update or rollback. The command confirms
2276	before cancelling; use `--yes` for an already reviewed automation. Cancellation
2277	prevents a delayed publication from taking effect. It does not change a box's
2278	release. A published activation stays held until its boxes complete or valid
2279	fencing evidence excludes them.
2280
2281	#### Instance layout
2282
2283	Every instance the activator runs used to live in one shared directory as one user, where it could read its neighbours' files. `layout migrate` moves them out: each instance gets its own directory and, from its next wake, its own mount and pid namespace, where it sees its own files and nothing else. An instance that is asleep is moved at once; one that is awake is left for up to an hour to go to sleep, then stopped briefly and moved. `--force` skips the hour. `--box` scopes the request to one box, so a fleet moves one box at a time with `carlos status --layout` showing each box's rows turn `nested`. Once a box's layout is `nested`, a newly provisioned instance on it starts out `nested`; it is never created in the shared directory first.
2284
2285	`layout migrate` is refused while any box in scope runs a platform release from before the layout, so a fleet is rolled forward first and migrated second.
2286
2287	What moves with an instance is everything in the box's data directory that carries its hostname: its database and the platform's files beside it, and anything the app made for itself under that name, such as a `<host>/` directory holding its keys or a `<host>-blobs` store. They land in the instance's directory under the same names, which is where an app that derives paths from its database path now looks. An entry that cannot be moved safely, such as a file hard-linked from outside the instance's own files, or a symlink, stops the move with its name in the reason, and the instance is held until it is cleared. A directory that carries no hostname, such as a `blobs` directory that several apps wrote into with the working directory `/`, is not moved: the box cannot tell whose it is. The agent log names those once a move has happened; an operator moves what an app owns into that app's instance directory by hand, and it comes along on any later revert or migration. A file named like another instance's platform files (`other.example.com.env`, `other.example.com.db`), including one left by an instance that has since been deleted or moved away, is never taken; the log names it. `revert` brings the same names back. What an app wrote relative to its working directory while nested (a `data/blobs` under its instance directory) is not among them: it stays in the instance's directory, where the flat app does not look, but nothing is lost. An instance that was moved before the platform carried entries along still has them in the shared directory; `carlos status --layout` and the agent log say so on every pass until an operator moves them.
2288
2289	You rarely need `migrate` at all. With no intent recorded, a box uses the nested layout by default as soon as its isolation is ready (the probe passes), and moves its instances on its own, with the same hour of grace for an awake one. `carlos status --layout` marks such a box `(nested by default)`, and says why a box the default has not reached is still flat. To keep box-by-box control, record `migrate --box` or `revert` first; a recorded intent always wins. A deployment that is not ready for this at all puts `CARLOS_LAYOUT_DEFAULT=flat` in each box's host.env.
2290
2291	`layout revert` moves them back. The order for rolling the platform back to a release from before the layout is: `layout revert`, wait until `carlos status --layout` shows every instance `flat`, then `system rollback`. The console will not move a channel pointer onto such a release while any box reports otherwise, and a revert is refused while a maintenance worker is attached to a box in scope. `layout status` prints the recorded intent; what each box has done is `carlos status --layout`.
2292
2293	```sh
2294	carlos system layout migrate --box kass-1
2295	carlos status --layout
2296	carlos system layout migrate
2297	carlos system layout revert --force
2298	carlos system layout status
2299	```
2300
2301	Ops account owners and `CARLOS_SYSTEM_OPERATORS`, like `update` and `rollback`.
2302
2303	A box is put on canary with `CARLOS_SYSTEM_CHANNEL=canary` in its host.env, and opts out of the nested instance layout by default with `CARLOS_LAYOUT_DEFAULT=flat` there.
2304	Boxes that have not reported a platform yet count as stable. `carlos system
2305	apply` is the root oneshot behind `carlos-system-update.service`; it is not
2306	for operators. The console is an app — update it with `carlos deploy --app
2307	console`.
2308
2309	#### Host lifecycle guard
2310
2311	`carlos system guard-arm` is a root-only deployment command that installs and
2312	arms the persistent host lifecycle checker. Supply `--policy`, `--identity`
2313	and `--enrollment`; console hosts also supply `--console` with the root-owned
2314	console executable and its `.prev` recovery image in place.
2315
2316	`carlos system guard-exec` is the root service entry point for checking an
2317	image before starting `apply`, `recover`, `edge` or `console`. Once armed,
2318	the local floor remains enforced independently of `CARLOS_HOST_CLAIM`.
2319	The member command remains `carlos ops host resolve`.
2320
2321	#### carlos system db-worker
2322
2323	`carlos system db-worker` runs on platform boxes only. systemd starts it for each forwarded database query. You never run it yourself.
2324
2325	#### carlos system claims
2326
2327	`carlos system claims` prepares the dev claim mechanics trial. Run `help` for the commands and required inputs. `run` executes one revision-checked phase from a protected plan on the trusted signing host; `inspect` returns the verified roster and its signed envelope. Box installers run as root. Add `--dry-run` to preview an operation.
2328
2329	`install-control` installs only instance-control and reloads systemd. `console-config` uses a separate console environment file and leaves host execution claims off. Neither installer starts its service.
2330
2331	Two `run` phases make the files `carlos system guard-arm` takes. `sign-policy` signs the armed policy for a floor and writes nothing to the bucket. `publish-identity` publishes a release's signed identity. It takes the router's hash from the release's signed checksum list, and the console's from the console app's signed release, chosen by its build. The console must be built with the release tag as its version, or the phase refuses it. Build it stripped, with -s -w, as releases are built. Each phase writes its file to `--out`, a new path, readable by root only. Pass these files to `guard-arm` exactly as written. A JSON tool would change the signed bytes.
2332
2333	Two more `run` phases make the backfill that `carlos system journal-migrate` and claim adoption read. Each takes the plan's `backfill.id`, and prints the signed plan as read back. `backfill-collect` freezes every instance record the bucket lists in a new signed plan. While that plan is in place, claim-mode record writes are refused. `backfill-adopt` runs once every roster box has published its census for the plan. It gives each host to the box that reported it, and a host no box reported is never adopted.
2334
2335	`backfill-done` is for a cohort with nothing to adopt. It closes a collecting plan as done, and claim-mode record writes are allowed again. It refuses unless the records are still the ones the plan froze, no host claim or app admission exists, and every roster box with a census key has published a census for the plan that lists no installation. A box with no census key is not asked, and is listed as unreported.
2336
2337	`install-console` copies the console build that the published identity names into `/opt/carlos/system/console/console` and `console.prev`, owned by root. It refuses a source whose bytes don't match the identity, and it doesn't touch `console.service`. Pass that path to `guard-arm --console`.
2338
2339	To replace an installed console, add `--upgrade --from <sha256>`, where the sha256 is the console that is running now. It refuses unless that build is what the box holds, then installs the new console and keeps the old one as `console.prev`. A box upgraded this way can't be armed again with `guard-arm` until `install-console` has been run without `--upgrade`, because `guard-arm` expects both copies to be the same build.
2340
2341	After starting the guarded console, run `console-report` on its box as root. It checks the signed roster, guard, systemd process and running executable, then records that process in the existing private box report. It refuses stale box evidence and leaves the report’s timestamp unchanged. Run it again after a console restart and immediately before floor publication; the next agent report may replace the process entry.
2342
2343	#### carlos system journal-migrate
2344
2345	Before starting claim-enabled services on a box with legacy journals, stop the edge and agent. Run `carlos system journal-migrate` as root with `CARLOS_HOST_CLAIM=1` and the deployment environment loaded. The command imports journals only when signed root stop evidence verifies them. It leaves the original files in place. After migration, root ignores changed legacy files and logs an alert. Keep the migration marker.
2346
2347	The migration census reads the backfill plan `backfill-collect` wrote. It reports each ordinary instance route on the box as that box's installation, so `backfill-adopt` can give the host to this box. If the window masks units by moving each unit file aside and linking `/dev/null` in its place, pass `--window DIR`, the window's directory, with the originals in `DIR/held`. The census then accepts those masks and no others.
2348
2349	#### Deployment-wide object scanning
2350
2351	```sh
2352	carlos system antivirus status
2353	carlos system antivirus enable
2354	carlos system antivirus disable
2355	```
2356
2357	`antivirus status` answers "is object scanning actually working here" without
2358	logging into a box. It reports **wiring** and **policy** separately, because
2359	they fail separately: an engine can be wired with scanning switched off, and
2360	scanning can be switched on with the engine unreachable. It also prints the
2361	signature-database version and warns when it is stale.
2362
2363	`enable` and `disable` are the switch. It lives in a record rather than in the
2364	unit file, so turning scanning on or off takes effect on the next tick with no
2365	restart — and `disable` stops every path, including `carlos store scan sweep`,
2366	not just the hourly pass. `enable` is refused when no engine is wired: a record
2367	claiming enabled with nothing behind it would report health for a deployment
2368	where no scan can run.
2369
2370	Deployment operators only (`CARLOS_STORE_OPERATORS`). Enabling scans nothing by
2371	itself — a member still requests scanning for their app and an operator still
2372	grants it per store.
2373
2374	##### Deployment-wide callback rotation
2375
2376	After configuring a new `CARLOS_SCAN_WEBHOOK_KEY`, keep the old key in `CARLOS_SCAN_WEBHOOK_KEY_PREVIOUS` during rotation and run:
2377
2378	```sh
2379	carlos system antivirus rotate-webhook-key
2380	```
2381
2382	The CLI delivers secrets in bounded batches until complete. It does not create or change the master key. Apps whose delivered secret changes restart. The current and previous master keys must differ; a previous key without a current key is refused.
2383
2384	#### Platform GeoIP
2385
2386	```sh
2387	carlos system geoip publish dbip-city-lite-2026-10.mmdb
2388	carlos system geoip status
2389	carlos system geoip off
2390	```
2391
2392	`publish` sends a MaxMind-format database, DB-IP Lite or GeoLite2, to the
2393	console. The console checks it, stores it in the deployment bucket, and every
2394	edge picks it up within 12 minutes with no restart. From then on, every
2395	request to every app carries the headers described under
2396	[Headers your app receives](#headers-your-app-receives), and analytics count
2397	page views by country. `publish` refuses a file that isn't a valid database,
2398	has no country data, or is a GeoLite2 database already past its 30 days. It
2399	also refuses one with under half the current database's nodes, which usually
2400	means a truncated file. `-force` overrides that last check only.
2401
2402	`status` shows the published database and every box: whether it runs a build
2403	that removes forged geo headers, and whether it has the current database.
2404	Its last line says `ready` only when every box does. Publish a database only
2405	after the whole fleet is ready, and tell apps about the headers only after
2406	that. Until every box has been updated, an older box passes a visitor's
2407	forged headers straight through.
2408
2409	A box that has been retired but still has an edges record keeps `status` from
2410	saying `ready`. Deleting its record (`control/edges/<label>.json`) retires it.
2411
2412	`off` removes the database. Every edge stops sending the headers within 12
2413	minutes.
2414
2415	**Keeping it current.** Set `CONSOLE_GEOIP_FEED` on the console and it fetches each new edition itself, checking once a day. `dbip-city-lite` is the free default and needs nothing else. For MaxMind, also set `CONSOLE_GEOIP_MAXMIND_ACCOUNT` and `CONSOLE_GEOIP_MAXMIND_KEY`: `geolite2-city` is MaxMind's free edition and `geoip2-city` its paid one. Each also comes as a `-country` edition, which is smaller and has no coordinates. MaxMind calls GeoLite2 unsuitable for commercial use, and its licence limits passing the data on, which serving it to apps may count as. Read it before choosing GeoLite2 for a commercial deployment.
2416
2417	**The licence is yours to keep.** DB-IP Lite is licensed under CC BY 4.0
2418	(https://db-ip.com/db/lite.php), so pages that show its data must credit it.
2419	The Analytics tab does this for you, and apps are told to do the same.
2420	GeoLite2 comes under MaxMind's GeoLite EULA
2421	(https://www.maxmind.com/en/geolite/eula), which also requires replacing the
2422	database within 30 days of a newer release. The platform enforces that by
2423	dropping a GeoLite2 database 30 days after it was built. Before you publish
2424	GeoLite2, apply the deployment-buckets module version that expires deleted
2425	GeoIP files after a day; the bucket is versioned, and otherwise old copies
2426	would linger.
2427
2428	Ops account owners and `CARLOS_SYSTEM_OPERATORS`.
2429
2430	#### The seeds floor
2431
2432	carlos system seeds-floor publish sets the first release whose boxes can place a seed. Publish it once every box runs that release. Each box arms it within the hour, and from then on a box never recovers to an older build. Until every box in a region has armed it, carlos instances create --from and carlos instances import refuse there.
2433
2434	```sh
2435	carlos system seeds-floor publish --min-release v0.27.0
2436	```
2437
2438	#### Legacy route assignments
2439
2440	carlos system legacy-routes assign says which account and app the routes an operator filed by hand belong to. A route added with carlos add in direct mode is filed under the operator account, so its app name alone does not say whose it is. An import takes over such a route only when an assignment maps its name to the importing app. A name has one assignment, and it cannot change while an import under it is in progress, from when its old process is marked stopped until it is committed or aborted.
2441
2442	```sh
2443	carlos system legacy-routes assign --name shop --to bac/shop
2444	```
2445
2446	#### Box and group roles
2447
2448	carlos system box role says whether new instances may be placed on a box. A box with the default role can take them. A legacy or platform box keeps the instances it has and takes no new ones: no new instance, relocation or move lands on it. carlos system group role sets the role of every box a region's group has enrolled, and of every box it enrolls later. Both work only on a console with region HA on.
2449
2450	Do not set a role until every box and console in the deployment runs a release that knows about roles. A box on an older release cannot read a roster that carries one.
2451
2452	```sh
2453	carlos system box role i-0123456789abcdef0 legacy
2454	carlos system group role ie default
2455	```
2456
2457	Ops account owners and `CARLOS_SYSTEM_OPERATORS`.
2458
2459	#### A box's identity record
2460
2461	carlos system box publish-record publishes the box's identity record to the deployment bucket, at control/boxes/<box>.json, where the host claim's enrollment reads it. The record is the box's EC2 instance identity document and AWS's signature over it, so AWS vouches for the box, not the box itself or you. The agent publishes it by itself only in a region with HA on. Anywhere else, run this once on the box as the agent user, with the box's host.env loaded. This is the only supported way to run it:
2462
2463	```sh
2464	systemd-run --wait --pipe -p EnvironmentFile=/etc/carlos/host.env -p User=carlos /opt/carlos/carlos system box publish-record
2465	```
2466
2467	It refuses to run as root, on a fleet box, and on a box with no S3 deployment bucket. It only ever creates the record, and only an S3 bucket can refuse to overwrite one. If a record is already there, it checks it the way enrollment does: AWS's signature, the account, the region, the instance and the box name. A valid record is kept as it is, even an older one written before the instance was stopped and started, and nothing is written. Any other record is refused, with what is wrong, and left alone. A valid record means only that the evidence is in place. Enrollment still makes its own checks before it accepts the box.
2468
2469	#### Boxes a region's group launches
2470
2471	In a region with HA on, the boxes its Auto Scaling group launches set themselves up with no hand steps. A box's start-up runs two commands as root before it starts the agent.
2472
2473	carlos system host-env write builds /etc/carlos/host.env from the deployment's parameters: SecureStrings in AWS SSM under /carlos/<deployment>/<region>/ and, for the values every region shares, /carlos/<deployment>/shared/. If a required value is missing, it stops and writes nothing.
2474
2475	carlos system install-channel installs the release the box's channel names, as both carlos and carlos.prev, once it passes the published host floor. A box that already runs a release is refused: carlos system update moves it.
2476
2477	With --policy FILE, the armed policy carlos system claims run sign-policy wrote, it installs before the host floor is published. The policy's floor stands in for the published one, and once a floor is published the two must be the same. It also arms a box that carlos system update has already moved to the channel's release: carlos is left as it is, and carlos.prev, the release before, is replaced by it, so guard-arm finds the release in both. A box on another release, or with an armed guard, is still refused.
2478
2479	With --policy, --identity FILE is needed too: the release identity carlos system claims run publish-identity wrote. Before a floor is published, the channel pointer carlos system update writes names no release tag, so the release is found through this file instead. It must be the identity published for its tag, and it must name the release and the artifact the channel names.
2480
2481	```sh
2482	carlos system host-env write --deployment prod --region ie
2483	carlos system install-channel
2484	```
2485
2486	The agent then joins. It publishes the box's identity record and hosts nothing. Once the console has checked with AWS that the new box belongs to the region's group, it invites the box. The box arms its host lifecycle guard, the console checks the group again and enrolls it, and the box restarts as a hosting box. Only then does the load balancer send it traffic.
2487
2488	carlos system host-env seed copies a box's existing host.env into those parameters, once per region. It writes the region's own values and any shared value not yet set. It refuses to change a shared value another region has set.
2489
2490	```sh
2491	carlos system host-env seed --deployment prod --region ie --kms-key alias/carlos-prod-config
2492	```
2493
2494	Seeding needs ssm:GetParametersByPath and ssm:PutParameter on the deployment's two parameter paths, and kms:Encrypt and kms:Decrypt on its configuration key.
2495
2496	### carlos move
2497
2498	Moves one instance to another region. The instance keeps serving while its
2499	data is copied, then goes offline for the cut. If the new region cannot start
2500	it, the move takes it back to where it was, on its own.
2501
2502	Moves are off unless the console runs with `CARLOS_MOVES_PHASE1` on. Against
2503	any other console, `carlos move` sends nothing. Run it as yourself, not as
2504	root.
2505
2506	```sh
2507	carlos move --host <host> --to <region> [--manual-cut] [--no-wait]
2508	carlos move status --host <host> [--wait]
2509	carlos move cut --host <host>
2510	carlos move abort --host <host>
2511	carlos move resolve --host <host> --retry
2512	carlos move resolve --host <host> --declare-lost <box>
2513	```
2514
2515	`carlos move`, `cut`, `abort` and `resolve` wait until the move ends unless
2516	you pass `--no-wait`. `carlos move status` reads the move once; with `--wait`
2517	it waits too, which picks a wait back up after a disconnect. `--manual-cut`
2518	stops once the move is ready, and `carlos move cut` starts the cut.
2519	`--app` and `--account` name the instance's app and account, as for
2520	`carlos instances`.
2521
2522	`--declare-lost <box>` declares the move's target box lost, for a box that
2523	will not come back. It needs the console's host claim. It reads the box
2524	through your own AWS credentials and refuses unless AWS shows it stopped or
2525	terminated. It says what the declaration loses, and you type the host name
2526	to confirm.
2527
2528	| exit code | meaning |
2529	|---|---|
2530	| `0` | the move finished, the console accepted a command given `--no-wait`, or `status` found the move still under way |
2531	| `3` | the move rolled back or was aborted |
2532	| `4` | the move needs an operator: it is stuck, waiting for a box, or ready for a manual cut, or its outcome was not recorded |
2533	| `5` | refused, by the console or before anything was sent |
2534	| `1` | the console could not be reached, or answered with an error |
2535
2536	Operators who are members of the instance's account.
2537
2538	`carlos move offline` moves one instance with planned downtime, one step at a time. It is for operators, works with host claims and Phase 1 moves off, and never acts on its own: each verb does one step and says which verb comes next.
2539
2540	```sh
2541	carlos move offline stage --host <host> --to <region>
2542	carlos move offline start --host <host> --to <region>
2543	carlos move offline seal --host <host>
2544	carlos move offline flip --host <host>
2545	carlos move offline open --host <host>
2546	carlos move offline finish --host <host>
2547	carlos move offline abort --host <host>
2548	carlos move offline status --host <host>
2549	```
2550
2551	The instance is offline from `start` until `open`. Until `open`, `abort` puts it back where it was with nothing lost. After `open`, moving it back is a new move.
2552
2553	`stage` is optional and comes before `start`. It copies the instance's data to the region it moves to while the instance keeps serving, and each time it runs it copies only what is new. The more it has copied, the less is left to copy while the instance is offline. Until `start`, `abort` removes the staged copy and the instance is never offline.
2554
2555	## On the host
2556
2557	These are what systemd starts. You will rarely type them, but knowing what
2558	they do explains most of what happens between a promote and a URL changing.
2559
2560	### carlos edge
2561
2562	The front door. TLS and ACME, and the proxy that maps a hostname to whatever
2563	is serving it. One per box, started by `carlos-edge.service`.
2564
2565	```sh
2566	carlos edge --dev --http :8080
2567	```
2568
2569	`--dev` serves plain HTTP with no TLS or ACME, which is how you run it on a
2570	laptop. `--fallback` redirects unknown hostnames somewhere (an apex marketing
2571	site, usually) instead of returning the plain 404.
2572
2573	### carlos agent
2574
2575	Edge, the hibernation activator, release adoption and the status tick, in one
2576	process per host. This is what a box actually runs; `carlos edge` alone is
2577	the proxy without any of the rest.
2578
2579	### carlos adopt
2580
2581	One release-adoption pass: read each route's channel, resolve what that
2582	channel points at, fetch it if the box does not have it, and swap the
2583	symlink. The hourly timer runs this verb.
2584
2585	```sh
2586	carlos adopt
2587	```
2588
2589	Adoption swaps a symlink, and on its own it does not restart anything. A
2590	long-running unit keeps serving the old build until something cycles it, and
2591	a hibernating tenant picks the new one up when it next wakes — so the order
2592	that works is promote, then adopt, then restart. A route that names its
2593	owning unit is the exception: adoption cycles that one itself, and rolls the
2594	adoption back if the restart fails, so the version header never claims a
2595	build the process is not serving.
2596
2597	`--restart-only` and `--config-only` are the narrower passes the agent asks a
2598	root-side unit to run for it, since the agent can request a pass but cannot
2599	write to `/etc` or restart units itself.
2600
2601	### carlos instance-control
2602
2603	The root half of isolated exec. `carlos-instance-control.socket` activates
2604	it when the agent connects; it starts, stops and sweeps the
2605	`carlos-instance*@` units for the agent, sets each host's memory tier on
2606	its slice before a start, and runs the isolation probe at start and every
2607	five minutes. It accepts a connection only from the edge unit's own main
2608	process. Not for operators: there is nothing to type.
2609
2610	It stays up once the first wake has started it, so it has to be restarted
2611	whenever the platform binary changes; `carlos system update` does that, and
2612	a hand roll must too. A box checks both ends: the service refuses to be
2613	ready while the binary it runs is no longer `/opt/carlos/carlos`, and the
2614	agent refuses to wake isolated instances while the service reports another
2615	build than its own, or none. Either way the reason in `carlos logs` ends
2616	with `restart carlos-instance-control.service`.
2617
2618	### carlos instance-exec
2619
2620	The first thing an isolated instance unit runs. It reads the launch file
2621	the agent wrote into the instance's directory and starts the host's live
2622	link with exactly the arguments and environment in it, refusing anything
2623	that is not the platform's own live link. It then stays on as the
2624	instance's first process: it passes stop signals on to the app, cleans up
2625	exited processes, and exits with the app's status, so an app with no
2626	signal handling still stops cleanly.
2627	`ExecStart=` of the four instance templates.
2628
2629	### carlos instance-exit
2630
2631	`ExecStopPost=` of the four instance templates. It leaves an advisory
2632	exit record in the instance's run directory from the four variables
2633	systemd provides; the agent proves termination from the unit's state and
2634	its cgroup, not from this file.
2635
2636	### carlos instance-probe
2637
2638	The isolation probe, run under the same namespace text as the instance
2639	templates. From inside it checks what a tenant would see: only its own
2640	directories, only its own pid, a read-only cgroupfs, no `/etc/carlos`, a
2641	symlink that is not followed, and a metadata service that a packet filter
2642	refuses. It prints a verdict; `carlos-instance-control` reads it and
2643	refuses to start isolated instances until every line passes.
2644
2645	### carlos ops
2646
2647	Box operations, run by a timer or by an operator standing on the box.
2648	`litestream` regenerates the replication config from the registry and the
2649	data directory, printing `changed` or `unchanged` so the caller knows whether
2650	to restart replication. `restore-verify` restores every configured replica
2651	from S3 and integrity-checks it. `hibernate-verify` does the same drill for
2652	the replicas the config does not name — hibernating instances, whose
2653	replication belongs to the activator — and publishes a per-host verdict to
2654	the deployment bucket so you can read it off-box.
2655
2656	```sh
2657	carlos ops restore-verify
2658	```
2659
2660	## Everything else
2661
2662	`carlos help` belongs to all three roles above.
2663
2664	### carlos help
2665
2666	Prints the command list, grouped the way this page is. It goes to stdout and
2667	exits 0, so `carlos help | less` works. A bare `carlos` with no arguments
2668	prints the same text on stderr and exits 2, because that one is a mistake.
2669