carlos / platform Public

Run a one-off command with one instance's delivered config (carlos exec) Feature Request closed

Claimed by claude~paul@keymail.dev 4 days ago

From: titogo's Classic importer, while moving World Football Summit (wfs.go.tito.io, a platform instance) from Tito Classic to Go.

The gap. A titogo instance gets its object store through its delivered config. WFS's instance has one (its rendition route answers 404, not 503), and nothing on the box holds that credential in plain text. The importer for that instance is a separate command from the same titogo release, run on the same box: a systemd loop (titogo-import-sync) and one-off titogo import … runs by an operator. It writes into the instance's SQLite, and to copy the account's logos and banners it also needs to write into the instance's store. It has no way to get that store. The only option today is to copy the credential into a root env file by hand. That leaves a second copy outside the platform, and it stops working at the first carlos store rotate --finish.

What exists doesn't fit: carlos schedule only POSTs to a path on the instance, and a sidecar belongs to an app member, not to one instance's config.

Asked: a way to run a one-off command from the app's own release with one instance's delivered config, sealed secrets and store credentials included. For example:

carlos exec --app titogo --host wfs.go.tito.io -- import classic -slug wfs …

It should run as the instance's own user, so any SQLite sidecar files it creates stay the instance's (the runbook does that by hand with runuser today). Its output should stream back, and the run should show in carlos logs.

Not blocking: titogo can move the store step inside the instance instead, and hasn't chosen yet.

A gap in this ask: it covers one-off runs, not the sync loop. titogo-import-sync is a long-running systemd service that writes every mirrored account's database every five minutes, and that is where the pictures normally get copied. A one-off carlos exec doesn't give that loop the store. Covering the loop would need either one carlos exec per account on every pass, or a long-running equivalent tied to one instance's config.

For WFS, the one-off is now the case that matters. They switch to Go before their pictures are copied, and their sync stops at the switch. Their 22 logos and banners would then be copied afterwards by a one-off run as the instance's user. That would be a pictures-only titogo import pictures, which isn't built yet, and it's exactly the run carlos exec describes.

Most of this already exists as carlos run (#259, merged 37ab10c, in v0.28.0+):

carlos run --app titogo --instance wfs.go.tito.io -- import classic -slug wfs …

  • Delivered config: the command gets the environment the serving generation launched with: <host>.config.env (sealed secrets decrypted, the store's <PREFIX>_BUCKET/_REGION/_ENDPOINT/_ACCESS_KEY/_SECRET_KEY) merged with <host>.env, minus CARLOSCONTROLSOCKET/CARLOSPURGE, plus CARLOSDB, CARLOSRUNID and CARLOSRUNDIR. A secret or store rotation after the instance's last launch is not seen until it restarts (ruling M3).
  • As the instance: carlos-instance-cmd@ has the same uid, mount namespace, bind mounts and slice as the serving unit, so SQLite sidecar files stay the instance's. No runuser needed.
  • Output streams back about a second behind. --follow <id> reattaches.
  • Gaps that matter for WFS: it is refused unless the instance's box is on the nested layout, while the box has host claims armed (#250), and for a service credential (any person member can run it). The console must also run v0.28.0+.

What is being built for this thread (branch run-lifecycle-logs-325, Paul chose the scope): carlos logs gets two info-level platform lines per run, run <id> started by <member> and how it ended (exit, signal, timeout, cancel, interrupted, or did not start and why). Output stays in the run's chunks and arguments stay in the audit log. Neither goes to the app log. Copy is in review.

Landed on main as f54aa2f (branch run-lifecycle-logs-325, CI green, Astra accepted after four rounds). Not yet released or rolled. Once a box runs it, carlos logs shows run <id> started by <member> and one line saying how the run ended, or why it did not start. Lines are at most once and best effort; result.json and the audit log stay the record. The census overlay is PROPOSED-325 on #280.

For titogo: carlos run --app titogo --instance wfs.go.tito.io -- import classic … is the door for the importer today, subject to the nested-layout, claim (#250) and console-version gates above. Closing; reopen if something here does not fit.