amadan
specs: carlos run (#259) and instances seeded from a database (#260, #261)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Human
Prompt
Can we pick up #259, #260, and #261 ?
From this commit's own message. The prompt ledger has no entries for it — which is what a squashed, rebased or amended commit looks like, and what every commit looks like in a repo without a ledger.
Summary
Two design specs, rev 1, from the design agreed with Paul in chat on 2026-10-08: carlos run (member-run one-off command as the instance, output via bucket chunks) and seed snapshots (instances create --from, and the import door that converts Tito's controld instances).
Agent notes
Rulings recorded in each spec's section 2. The 24h lifecycle rule mentioned in chat became a console sweep: runs live under each app's subtree, which a prefix rule cannot match.
2 files changed,
+705
−0
docs/superpowers/specs/2026-10-08-carlos-run-design.md+315 −0docs/superpowers/specs/2026-10-08-seeded-instances-design.md+390 −0
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-08-carlos-run-design.md b/docs/superpowers/specs/2026-10-08-carlos-run-design.md| new file mode 100644 |
| index 000000000..92d23e413 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-08-carlos-run-design.md |
| @@ -0,0 +1,315 @@ |
| +# `carlos run`: a member runs an app command against one instance (#259) |
| + |
| +**Status:** rev 1, 2026-10-08. Design agreed with Paul in chat the same day; |
| +this document is for his review, then Astra's. |
| + |
| +**Discussion:** amadan carlos/platform #259. Companion spec: |
| +`2026-10-08-seeded-instances-design.md` (#260, #261). |
| + |
| +## 1. The problem |
| + |
| +Every one-off command against a live instance today goes over |
| +`aws ssm send-command` as root on the box. On 2026-10-06 a titogo session |
| +imported 22,462 check-ins that way (titogo PR #4048): it ran the app's own |
| +binary against the instance's database, read a credential from the app's |
| +config, and staged the input file through presigned URLs. The canary recipe |
| +(`titogo add`, `auth grant`, `seed`) has the same shape. |
| + |
| +Two members of Tito's team can sign in to the console as members of `tito` |
| +but cannot do any of this without root on a box every tenant shares. The |
| +platform has no door for "run this app's command against that instance". |
| + |
| +## 2. Rulings (Paul, 2026-10-08) |
| + |
| +1. **Who may:** any person who is a member of the account, the same rule as |
| + `carlos db pull`. Service credentials are refused. A member can already |
| + read every instance's database and publish to non-stable channels, so |
| + this grants nothing new. |
| +2. **Output transport:** the run's own chunks in the deployment bucket, |
| + polled by the CLI through the console. No new network path; the console |
| + still never connects to a box. |
| +3. **Platform-run instances only.** controld-run (`titogo@`) instances are |
| + converted (#261) rather than served. |
| + |
| +## 3. What it does |
| + |
| +``` |
| +carlos run --app <app> --instance <host> [--file <path>]... [--timeout 30m] -- <arg>... |
| +``` |
| + |
| +- Runs the release the instance follows, as `/opt/carlos/live/<host> |
| + <arg>...`. The caller never names the binary. |
| +- Runs as the instance: same uid, same namespace, same bind mounts, same |
| + slice, same environment. Never root, never the agent. |
| +- Streams stdout and stderr to the terminal (about one second behind) and |
| + exits with the command's own status. |
| +- Records who ran what, where and how it ended. |
| + |
| +**Defaults** (agreed in chat): |
| + |
| +| | | |
| +|---|---| |
| +| hibernated instance | woken first, so its database is restored and replicating | |
| +| live instance | left running; the command opens the same database beside it (WAL) | |
| +| stdin | none; argv only | |
| +| Ctrl-C | cancels the run on the box | |
| +| `--timeout` | 30 minutes by default, 6 hours at most | |
| +| `--file` | repeatable; 64 MB in total | |
| +| concurrency | one run at a time per instance | |
| +| layout | refused unless the instance is nested | |
| + |
| +**Not in this version:** |
| + |
| +- `--version`. The trampoline execs only `/opt/carlos/live/<host>` |
| + (`internal/instanceexec/exec.go:72`), and the control service binds that |
| + link to the instance's own build. Running another build needs a second |
| + exe link and its own review. |
| +- `--stopped` (stop, run, start). |
| +- `--record` (keep the output). |
| +- An interactive shell. `run` is argv only, which keeps every run |
| + auditable. |
| + |
| +## 4. The command's contract |
| + |
| +What the app's binary sees, for the `docs/site/cli.md` section "What an |
| +instance is started with": |
| + |
| +- **argv:** exactly what followed `--`, after the binary. The platform |
| + appends nothing, unlike a seed's `--db` (`seedArgv`). |
| +- **Environment:** the same one a seed gets (`seedVars`, |
| + `backend_exec.go:1359`): the app's config bundle with `<host>.env` on |
| + top, `CARLOS_HOST`, `CARLOS_REGION`, `CARLOS_ADMIN_TOKEN`, and no control |
| + socket. Plus: |
| + |
| +| Variable | Value | |
| +|---|---| |
| +| `CARLOS_DB` | the instance's database path, the one `--db` names when the instance is started | |
| +| `CARLOS_RUN_ID` | this run's identifier | |
| +| `CARLOS_RUN_DIR` | a directory holding the `--file` uploads under their base names. Present even with no files; deleted after the run | |
| + |
| +- **Working directory:** the instance's data directory, as for every |
| + template (#256 item D). |
| +- **Signals:** SIGTERM on cancel or timeout, SIGKILL 20 seconds later |
| + (`KillMode=mixed`, `TimeoutStopSec=20`, as the other templates). |
| + |
| +Secrets reach the command only through its environment, never in argv the |
| +platform builds, and the platform never prints the environment. |
| + |
| +## 5. The flow |
| + |
| +### 5.1 Request (CLI → console) |
| + |
| +`POST /api/cli/apps/{account}/{app}/instances/{host}/runs`, multipart: a |
| +`meta` part (`argv`, `timeout`, file names and sizes) and one part per file. |
| +The console streams file parts straight to the bucket (the F8 rule: bytes |
| +go through the console, never a presigned URL) under a request-wide |
| +`http.MaxBytesReader` of 64 MB plus the meta allowance. |
| + |
| +The console refuses, in this order, each with its own status and message: |
| + |
| +1. A service credential (`cliRoute` with the zero `svcrole.Set`). |
| +2. Not a member of the account: the byte-identical 404 `apiAccountApp` |
| + gives. |
| +3. Budget (429), on the instance verbs' per-token window. |
| +4. No record for the host, or a record of another app: 404. |
| +5. The record is not platform-run exec, or is pooled, or is placed on a |
| + customer's fleet: 409. |
| +6. A move is in flight (`move` state other than none): 409. |
| +7. A run is already open on the instance: 409, naming its id and caller. |
| +8. argv empty, longer than 256 arguments or 64 KB in total, or holding a |
| + NUL byte; timeout outside 1s to 6h; files over the cap or with names |
| + that are not a plain base name: 400. |
| + |
| +It then writes, under the app's own subtree (`release.AppPrefix`, so the |
| +owner gate maps onto it and app trash and purge sweep it): |
| + |
| +- `runs/<id>/in/<name>` for each file; |
| +- `runs/<id>/request.json`: host, argv, timeout, file list with sizes and |
| + sha256, caller, requested-at; |
| +- `runs/index.json`: the app's open runs (id, host, state), updated by |
| + conditional PUT on its ETag; |
| + |
| +bumps `control/serial`, writes the audit row (§6), and answers `202` with |
| +the run id. |
| + |
| +### 5.2 Pickup (box) |
| + |
| +`runs/index.json` follows `carlos restart`'s pattern (`internal/host/ |
| +restart.go`): it is read only when the serial moves, and a box acts only |
| +when the index's ETag differs from the one it last saw. A box with nothing |
| +happening pays nothing beyond its existing serial GET. |
| + |
| +A box acts on a run only for a host whose exec row it owns: the registry |
| +row is on this box and, with the host lifecycle claim on, this box holds |
| +the claim. Every other box ignores it. The owning box: |
| + |
| +1. Writes `runs/<id>/picked.json` (box, time). The console reports |
| + "not picked up" if this is absent 60 seconds after the request. |
| +2. Refuses unless `activator.Classify` answers `LayoutNested` for the row |
| + and `nestedGate` passes. A refusal is a `result.json` with |
| + `refused: "flat"`, never a fallback to running the command as the |
| + agent. |
| +3. Wakes the instance if it is hibernated, through the ordinary wake path, |
| + so the database is restored and replication is running. A wake failure |
| + is a `result.json` with `refused: "wake"` and the wake's error. |
| +4. Takes a **run hold** on the instance: while it is held, hibernation |
| + does not stop the instance and a move request is refused. Without it, |
| + a hibernation during the run would stop replication and lose the |
| + command's last writes. |
| +5. Downloads the files into |
| + `/data/instances/<host>/.carlos-run/<id>/`, checking each against the |
| + size and sha256 in `request.json`, mode 0700, owned by `carlos`. |
| + |
| +### 5.3 The unit |
| + |
| +A fourth template, `carlos-instance-cmd@.service`, copied from |
| +`carlos-instance-seed@.service` with every sandbox line unchanged and a |
| +test that fails if the two ever differ outside the lines below: |
| + |
| +- `Description=carlos exec-backed instance %I (cmd)` |
| +- `ExecStart=/opt/carlos/carlos instance-exec cmd %I` and |
| + `ExecStopPost=/opt/carlos/carlos instance-exit cmd %I` |
| +- `TimeoutStartSec=6h5m`, a backstop past the 6-hour cap; the agent |
| + enforces the run's own timeout |
| +- `StandardOutput=file:/run/carlos/logs/%I.cmd.sock` and |
| + `StandardError=file:/run/carlos/logs/%I.cmd-err.sock`, two listeners so |
| + the streams stay separate |
| + |
| +`instancectl` gains `RoleCmd = "cmd"` (`internal/instancectl/names.go`; |
| +`run` is the serving role) and `UnitPrefixCmd = "carlos-instance-cmd@"`, so |
| +the control service starts, stops and sweeps it like the other three. Its |
| +`POST /cmd` starts the unit and answers at once, unlike `/seed`, which |
| +blocks for at most 150 seconds; the agent then waits on the unit with |
| +bounded `GET /cmd/{host}` calls that answer `{active, InvocationID, |
| +Result, ExecMainStatus, Code}`. The agent writes `launch.cmd.json` through |
| +`writeLaunch`, with argv `[/opt/carlos/live/<host>, <arg>...]` and the |
| +environment in §4. |
| + |
| +### 5.4 Output |
| + |
| +The agent reads the two sockets and writes |
| +`runs/<id>/out/<seq>-<stream>` (stream `o` or `e`; seq zero-padded, |
| +shared across both streams so the CLI can interleave them in the order the |
| +agent read them) every second or every 256 KB, whichever comes first. |
| +Total output is capped at 64 MB; past the cap the agent stops writing |
| +chunks and writes one `truncated` marker, and the command keeps running. |
| + |
| +When the unit becomes inactive the agent writes `runs/<id>/result.json`: |
| + |
| +```json |
| +{"exit": 0, "signal": "", "timed_out": false, "cancelled": false, |
| + "started": "...", "ended": "...", "box": "...", "chunks": 17, |
| + "truncated": false, "refused": ""} |
| +``` |
| + |
| +then deletes the run directory and releases the run hold. |
| + |
| +### 5.5 Following (CLI ↔ console) |
| + |
| +`GET …/runs/{id}?after=<seq>` answers the chunks after `seq` (bounded per |
| +response) and `result.json` once it exists. The CLI polls it every second, |
| +writes `o` chunks to stdout and `e` chunks to stderr, and exits with: |
| + |
| +| case | exit | |
| +|---|---| |
| +| command exited | its own status | |
| +| command killed by a signal | 128 + the signal number | |
| +| timed out | 124 (`timeout(1)`'s) | |
| +| refused by the box (flat, wake) or not picked up in 60 s | 3 | |
| +| refused by the console | 2 | |
| + |
| +### 5.6 Cancel and timeout |
| + |
| +Ctrl-C makes the CLI send `POST …/runs/{id}/cancel`. The console sets |
| +`cancel` on the run in `runs/index.json` and bumps the serial; the owning |
| +box stops the unit. A second Ctrl-C exits the CLI without waiting; the |
| +cancel still happens. The agent stops the unit itself at the run's |
| +timeout. Both end in an ordinary `result.json`. |
| + |
| +If the CLI goes away without cancelling, the run continues to its end or |
| +its timeout. `carlos run --follow <id>` attaches to an open or finished run |
| +and prints everything not yet acknowledged. |
| + |
| +### 5.7 Cleanup |
| + |
| +- When the CLI has the result it sends `POST …/runs/{id}/ack`; the console |
| + deletes `runs/<id>/` and drops the run from the index. |
| +- A console sweep (the existing deadline-sweep loop) deletes any run whose |
| + result is more than 24 hours old, and closes any run with no result |
| + whose timeout has passed by an hour, writing the audit row's end as |
| + `lost`. This replaces the 24-hour bucket lifecycle rule named in chat: |
| + the runs live under each app's subtree, which a prefix rule cannot |
| + match, and the sweep needs no infrastructure change. |
| +- The box sweeps `.carlos-run/` directories with no open run at agent |
| + start. |
| + |
| +## 6. The record |
| + |
| +Two audit rows through `Server.auditActivity`, target the instance: |
| + |
| +- `run-start`: caller, app, host, argv, file names and sizes, the release |
| + version the instance follows, run id. |
| +- `run-end`: run id, exit, signal, timed out, cancelled, started, ended, |
| + box; or `lost`. Written by the console the first time it reads |
| + `result.json`, whether from the CLI's poll or the sweep. |
| + |
| +Output is never in the audit log. argv is: a member who puts a secret in |
| +argv has put it in the log, and `docs/site/cli.md` says so and says to pass |
| +secrets through `--file`. |
| + |
| +## 7. Security |
| + |
| +- **Reach.** The command has exactly the instance's reach, which is why it |
| + is refused on a flat box: there it would have the agent's. |
| +- **No widening for members.** A member can already pull the database and |
| + publish a build to a non-stable channel that the instance runs; `run` |
| + executes only the build the instance follows. |
| +- **Box-written results are box-forgeable.** Any box can write anywhere in |
| + the deployment bucket today (#171); a compromised box could forge |
| + another box's output or result. That is the existing record-authority |
| + boundary, not a new one, and `run` adds no path from a forged result to |
| + any action. |
| +- **File names** are plain base names checked at the console and again by |
| + the agent, which writes with `instancefs` (no symlink is followed). |
| + |
| +## 8. Errors the member sees |
| + |
| +Each is its own message, and every one names the instance: |
| + |
| +- not a member, or no such instance; |
| +- not platform-run, pooled, or on a customer's fleet; |
| +- moving; |
| +- a run already open (with its id and who started it); |
| +- the box is not nested ("this box has not moved to the isolated layout |
| + yet"); |
| +- not picked up within 60 seconds; |
| +- the wake failed (with the wake's error); |
| +- timed out; |
| +- file too large, or output truncated. |
| + |
| +All CLI and console strings go through copy review before they are written |
| +into the code. |
| + |
| +## 9. Testing |
| + |
| +- Console door: each refusal in §5.1 in order; the member-only rule; the |
| + audit rows; the index CAS under two concurrent requests; cancel, ack and |
| + the sweep. |
| +- Box: pickup only on the owning box; the flat refusal; the wake; the run |
| + hold against hibernation and a move request; file checks; chunking, |
| + ordering and the output cap; timeout; cancel; result. Against a fake |
| + control service and an in-memory bucket. |
| +- Units: the cmd template equals the seed template outside the named lines. |
| +- `instancectl`: `RoleCmd` parsing, `/cmd` start and status, sweep. |
| +- CLI: interleaving, exit codes, Ctrl-C once and twice, `--follow`. |
| +- On the dev box once built: a real run against a nested instance, |
| + including a write the instance then serves, and a run across a |
| + hibernation timer. |
| +- `GOOS=darwin` and `windows` builds stay green: the unit and control code |
| + is linux-only behind build tags with stubs. |
| + |
| +## 10. Rollout |
| + |
| +Ships in the next release. It does nothing on a box until a member runs it, |
| +and refuses on a flat box, so no switch is needed. Tito's boxes need the |
| +nested layout before Doc and Vicky can use it (#247). |
diff --git a/docs/superpowers/specs/2026-10-08-seeded-instances-design.md b/docs/superpowers/specs/2026-10-08-seeded-instances-design.md| new file mode 100644 |
| index 000000000..45f21b24f |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-08-seeded-instances-design.md |
| @@ -0,0 +1,390 @@ |
| +# Instances that start from a database: `--from` (#260) and the import door (#261) |
| + |
| +**Status:** rev 1, 2026-10-08. Design agreed with Paul in chat the same day; |
| +this document is for his review, then Astra's. |
| + |
| +**Discussions:** amadan carlos/platform #260 and #261. Companion spec: |
| +`2026-10-08-carlos-run-design.md` (#259). |
| + |
| +## 1. The problem |
| + |
| +Two requests from Tito, one mechanism. |
| + |
| +- **#260, a canary with real data.** The commonest ask from Tito's team is |
| + "make me a canary with this account's real data". Today that takes root |
| + over SSM: copy the file, chown, register, grant. `carlos db pull` gives a |
| + member a read-only copy on their laptop, but there is no way back in: |
| + `instances create` cannot start from data, and `Manifest.Seed` is a |
| + first-boot argv (`provision.go:233`), not data. |
| +- **#261, converting controld instances.** About 71 `titogo@<host>` units on |
| + carlos-tito-ie run outside the platform, each behind an addr row. Paul |
| + ruled on 2026-10-08 that every one converts to platform-run and nothing |
| + new is built for controld instances. The import door that conversion needs |
| + was designed in `2026-09-24-instance-moves-for-every-app-design.md` §9 |
| + (gap G9, on the unmerged `docs-instance-moves-design` branch). The moves |
| + spec that replaced it (`2026-10-02-move-any-app-design.md`) dropped it: |
| + its "adoption" is the host claim taking over legacy platform records, not |
| + importing an instance the platform never ran. |
| + |
| +Both are "create an instance whose database starts as this one". |
| + |
| +## 2. Rulings (Paul, 2026-10-08) |
| + |
| +1. **One mechanism, two sources.** A sibling's replica (`--from`) and an |
| + uploaded snapshot (import) both become one object the box places on |
| + first wake. |
| +2. **Uploads go through the console**, not a presigned URL. This departs |
| + from §9 of the 09-24 design, on the console's own rule (F8, |
| + `apidb.go:299`, `internal/edge/storedoor.go`): bytes pass through the |
| + console, and nothing a request supplies picks a bucket or key. |
| +3. **`--from` stays in the source's region.** Moving to another region is |
| + `carlos move` afterwards, as §9 said for adoption. |
| +4. **The new instance gets its own environment** (never the source's |
| + per-host one) and **an empty private store**. |
| +5. **Who may, for both doors:** any person who is a member of the account, |
| + plus service credentials with the `instance` role. "This is day-to-day |
| + operator stuff ... this needs to be something developers can do." |
| + Stricter permissions may come later. |
| +6. **The swap accepts an operator-filed row** (§6.4): Tito's addr rows are |
| + filed under the deployment's operator account (G17), so a strict |
| + same-account check would refuse all 71. |
| +7. **Platform-run only, nested only.** Both doors refuse unless the target |
| + box is nested. |
| + |
| +## 3. The shared step: a seed snapshot |
| + |
| +### 3.1 The object |
| + |
| +`seeds/<host>/snapshot.db` in the home region's data bucket, beside the |
| +`instances/` and `moves/` prefixes. The console writes it; the owning box |
| +reads and deletes it. The console's region is the home region, the only |
| +one whose data bucket it may write (`resolveReplica`'s residency rule, |
| +`apidb.go:54`), so both doors refuse a record homed elsewhere with the |
| +existing `ErrResidency` message. |
| + |
| +### 3.2 The record |
| + |
| +`provision.Record` gains: |
| + |
| +```go |
| +Seed *SeedSource `json:"seed,omitempty"` |
| + |
| +type SeedSource struct { |
| + Kind string `json:"kind"` // "replica" | "import" |
| + From string `json:"from,omitempty"` // replica: the source host |
| + TXID string `json:"txid,omitempty"` // replica: the TXID restored |
| + SHA256 string `json:"sha256"` // of snapshot.db |
| + Bytes int64 `json:"bytes"` |
| + State string `json:"state"` // see below |
| + By string `json:"by"` // the caller |
| + At time.Time `json:"at"` |
| + // import only: |
| + Replaces string `json:"replaces,omitempty"` // the addr the swap replaces |
| + Reason string `json:"reason,omitempty"` // set with state refused |
| +} |
| +``` |
| + |
| +States: `awaiting-upload` → `uploaded` → `ready` → `placed`, or `aborted`, |
| +or `refused` (the box would not swap; `Reason` says why). |
| +A `--from` record is written straight at `ready`. Reconcile skips a record |
| +whose seed is not `ready` or `placed`. The box sets `placed` after the |
| +first wake succeeds. `Seed` stays on the record once placed: it is the |
| +provenance `instances list` shows. |
| + |
| +Old boxes must not mint a seeded record as an empty instance. The record |
| +is written only when every box in the region reports a release that |
| +understands `Seed` (the protocol-floor check the moves series uses, |
| +`moves_protocol`), otherwise the door answers 409 naming the boxes. |
| + |
| +### 3.3 The box: placing the snapshot on first wake |
| + |
| +One more fallback in `ExecBackend.Restore` (`backend_exec.go:611`), beside |
| +the staging fallback (`:633-645`), and the same step in the claim path's |
| +restore of a cold admission with no baseline (`claimwake.go`, `s.Restore`): |
| + |
| +1. A warm local database, or any replica of the host's own: the seed is |
| + not used. A host with its own replica has already been placed. |
| +2. The record's seed is `ready`: read `seeds/<host>/snapshot.db`, check its |
| + length and sha256 against the record, then `lease.CheckDB` (integrity |
| + and at least one table). |
| +3. Put it at the row's database path through `instancefs`, owned by the |
| + instance, in the nested tree. |
| +4. Start replication. The first upload is generation one of the host's own |
| + lineage: nothing of the source's lineage is copied. |
| +5. After the first upload succeeds: set the record's seed to `placed`, |
| + delete `seeds/<host>/`. |
| + |
| +Any failure in steps 2 to 4 fails the wake, with the reason in the box log |
| +and the `carlos logs` sink. It **never** falls through to an empty |
| +database and the manifest's seed command: a seeded instance that came up |
| +empty would look healthy and be wrong. The manifest's seed command is not |
| +run for a seeded instance at all; the data already exists. |
| + |
| +The snapshot is deleted only after the host's own replica exists, so a |
| +crash anywhere before that retries from the same snapshot. |
| + |
| +## 4. `--from`: a copy of a sibling (#260) |
| + |
| +``` |
| +carlos instances create --app <app> --host <new> --from <host> [--at <txid|time>] [--environment <env>] [--channel <c>] |
| +``` |
| + |
| +The console: |
| + |
| +1. Applies `instances create`'s existing checks to the new host. |
| +2. Resolves the source exactly as `db pull` does (`dbGate`): same app, same |
| + account, a record, not pooled, not on a fleet, homed in this region, |
| + and, with the host claim on, read through a versioned claim. |
| +3. Refuses a source and a target of different apps or accounts (409): |
| + customer data never leaves its account by this door. |
| +4. Restores the source's replica at its latest TXID, or at `--at`, with |
| + `dbdoor.RestoreTo` into a private 0700 directory under the console's |
| + own scratch, and runs `CheckDB`. This is the code `db pull` runs on the |
| + member's laptop, run here instead; it never reads the live file. |
| +5. Refuses a restored database over 4 GiB (413). The console needs scratch |
| + for one restore at a time per app; a second concurrent `--from` for the |
| + same app waits on a per-app semaphore, and one beyond a deployment-wide |
| + limit of two is refused with 429. |
| +6. Writes `seeds/<new>/snapshot.db`, then the record with seed `ready`, |
| + kind `replica`, the source and the TXID, then deletes its scratch. |
| + |
| +**Environment.** The new instance gets its own: `--environment` if given, |
| +otherwise what a plain `create` derives (the app's default, or the |
| +host-derived private-storage environment). Never the source's per-host |
| +environment, which carries host-keyed values such as the origin and the |
| +cookie domain (the warning in `instances_cli.go:235-245`). |
| + |
| +**Storage.** An app with private storage gets the new host's own empty |
| +store, as any `create` does. Copying the source's store is not offered in |
| +this version. |
| + |
| +## 5. The import door (#261) |
| + |
| +``` |
| +carlos instances import --app <app> --host <host> --file <snapshot.db> --replaces <addr> [--environment <env>] |
| +``` |
| + |
| +The CLI drives the four calls below. Titogo's conversion tool makes the |
| +same calls with its service credential. |
| + |
| +### 5.1 Begin |
| + |
| +`POST /api/cli/apps/{a}/{app}/instances` with |
| +`"import": {"sha256": "...", "bytes": N, "replaces": "<addr>"}` and the |
| +ordinary create fields. |
| + |
| +The console checks: |
| + |
| +- the caller (§2 ruling 5); |
| +- the host is not already a platform record. The console cannot see a |
| + box's registry, so the addr row's existence and shape are checked by the |
| + box at the swap (§5.4), which is the authority; |
| +- `bytes` is at most 4 GiB. |
| + |
| +It writes the record with seed `awaiting-upload` and answers `201`. No box |
| +mints anything for it yet. |
| + |
| +### 5.2 Upload |
| + |
| +`PUT …/instances/{host}/import`, streamed. The console hashes as it |
| +writes, in parts of 64 MB, to `seeds/<host>/snapshot.db`. A length or hash |
| +that differs from the record is refused (422), and the partial object is |
| +deleted. On success the seed is `uploaded`. |
| + |
| +The snapshot must be consistent and taken after the app stopped the |
| +instance (`VACUUM INTO` after its supervisor confirms the process is |
| +down). The console cannot see that. It can only check that the file is a |
| +SQLite database that passes `CheckDB`, which it does before marking the |
| +seed `uploaded`. |
| + |
| +### 5.3 Stopped |
| + |
| +`POST …/instances/{host}/import/stopped` is the app saying its own process |
| +is down. The seed becomes `ready`, and the console bumps the serial. |
| + |
| +Until then the app's own addr row keeps answering, so the app's router |
| +must answer 503 for a stopped instance. For a small database that window |
| +is a minute or two, and it is the whole cost of adopting an instance. |
| + |
| +### 5.4 The swap (reconcile) |
| + |
| +On the owning box, reconcile meets a record with seed `ready` and |
| +`Replaces` set, and an existing row for the host. |
| + |
| +Today an unprovisioned row of another shape is left alone with "exists but |
| +is not the route this record describes" (`reconcile.go:1158`, `:1287`). |
| +That stays the rule for every other case. This one case swaps instead, in |
| +one registry transaction, and only if all of these hold: |
| + |
| +- the row is an addr row (`Kind` addr) and its `Addr` equals `Replaces`; |
| +- the row is not provisioned by another record; |
| +- the row's app equals the record's app; and |
| +- the row's account is either the record's account, or the deployment's |
| + operator account (G17, §6.4). |
| + |
| +The addr row becomes the exec row this record describes, filed under the |
| +record's account. With the host claim on, the swap is the record's first |
| +claim acquisition: the claim CAS happens before the registry write, as for |
| +any create, and a refused CAS leaves the addr row untouched. |
| + |
| +If any condition fails, the box leaves the row alone, sets the seed to |
| +`refused` with the condition that failed, and mints nothing. The app sees |
| +that through `GET …/instances/{host}` (or `carlos instances list`), and |
| +abort then deletes the record. A swap that has not happened within five |
| +minutes of `stopped` is shown as pending with the last box that looked. |
| + |
| +The edge picks up the new row on its next reload. The next request wakes |
| +the instance, and §3.3 places the snapshot. |
| + |
| +### 5.5 Abort |
| + |
| +`POST …/instances/{host}/import/abort` undoes the import, which is how |
| +titogo rolls back automatically when `verify` fails: |
| + |
| +- **Before the swap:** delete the record and `seeds/<host>/`. |
| +- **After the swap:** mark the seed `aborted`. Reconcile on the owning box |
| + stops the instance and swaps the exec row back to the addr row it |
| + replaced (the record still names it), in one transaction. Then it |
| + releases the claim, deletes the record, and deletes the host's |
| + `instances/` replica and `seeds/<host>/`. |
| + |
| +Abort is refused once the seed is `placed` and the instance has served a |
| +request that was not the platform's own `verify`. After that, the instance |
| +may hold writes the old process never saw, and `carlos instances delete` |
| +is the honest verb. |
| + |
| +### 5.6 Verify |
| + |
| +`verify` (moves spec §6.5) is how titogo knows the adopted instance |
| +serves. Nothing new is needed here: once the exec row is live, the app |
| +calls its own verify path through the instance, and aborts if it fails. |
| + |
| +## 6. Who may, and what is checked |
| + |
| +### 6.1 Callers |
| + |
| +- People who are members of the account. |
| +- Service credentials with the `instance` role (`svcrole`), checked per |
| + route by `cliRoute`. |
| + |
| +The same rule applies to `--from`, to all four import calls and to abort. |
| +Tighter permissions can come later. |
| + |
| +### 6.2 Same app, same account |
| + |
| +- `--from` copies only between instances of one app in one account. |
| +- An import creates an instance of the app the route already belongs to. |
| + It cannot claim another app's host. |
| + |
| +### 6.3 Layout |
| + |
| +Both doors refuse at the console unless the region's layout report shows |
| +the target box nested (`apistatuslayout.go`). The box refuses again in |
| +§3.3 unless `Classify` answers `LayoutNested` for the row. On a flat box a |
| +seeded instance would be one uid among every tenant, and for Tito's |
| +controld instances it would lose the `0700 titogo` protection that is the |
| +only thing keeping them safe today (#126). |
| + |
| +### 6.4 The operator-account exception (G17) |
| + |
| +A direct-mode `carlos add` files a route under the deployment's operator |
| +account, not the app's own. Tito's rows are `account=bab` for app |
| +`bae/titogo`. The swap accepts that row when its app name matches, and |
| +files the new row under the app's account, which fixes G17 for every |
| +converted host. |
| + |
| +It accepts nothing wider: |
| + |
| +- not a row of another app; |
| +- not a row filed under a third account; |
| +- not a row that is not addr-backed; |
| +- not a row whose addr has changed since `Replaces` was recorded. |
| + |
| +## 7. What a member sees |
| + |
| +- **`carlos instances list`** marks a seeded instance: |
| + - "seeded from <host> at txid N" for `--from`; |
| + - "imported <first 12 of sha256>" for an import; |
| + - "import awaiting upload" or "import awaiting stop" while those are |
| + pending. |
| +- **Audit rows:** |
| + - `instance-seed` names the source host and TXID, or the sha256 and |
| + bytes, and the caller; |
| + - `import-stopped`, `import-swapped` and `import-aborted` each record |
| + their step. |
| +- **Errors** each have their own message: |
| + - other app or other account; |
| + - other region; |
| + - box not nested; |
| + - too large; |
| + - hash or length mismatch; |
| + - not a database; |
| + - no addr row to replace; |
| + - addr changed; |
| + - row belongs to another app; |
| + - boxes below the protocol floor; |
| + - abort refused after serving. |
| + |
| +All CLI and console strings go through copy review before they are written |
| +into the code. |
| + |
| +## 8. Security |
| + |
| +- **Data never leaves its account.** Neither door crosses an account or an |
| + app. |
| +- **Uploads are pinned.** An import is pinned to one record, one length |
| + and one hash. The console checks the hash as it writes, and the box |
| + checks it again before it trusts the file. |
| +- **No standing identity gains a read.** The console already reads the |
| + home region's replicas for `db pull`. Boxes read only their own |
| + region's bucket, and `seeds/` is in it. |
| +- **A forged record is still the main risk.** A seed is a record field, |
| + and #171 means any compromised box can forge a record. A forged seed |
| + could make a new instance start from data the forger chose, though only |
| + data that is already in that bucket. This is the same exposure the |
| + moves series names as G13, and #171's fix bounds both. |
| +- **The swap only narrows the "leave it alone" rule.** It narrows it for |
| + a row that this record names by its exact addr, that belongs to the |
| + same app, and that is filed under the app's own account or the |
| + operator's. Every other mismatch is left alone, as today. |
| + |
| +## 9. Testing |
| + |
| +- **Box:** |
| + - placing the snapshot, and each failure (length, hash, `CheckDB`) failing |
| + the wake without ever falling back to an empty database; |
| + - a crash before the first upload retrying from the snapshot; |
| + - the snapshot deleted only after the first upload; |
| + - in both the plain path and the claim path. |
| +- **Reconcile:** |
| + - the swap's four conditions, each refused on its own; |
| + - the operator-account case; |
| + - a refused claim CAS leaving the addr row; |
| + - abort after the swap restoring the addr row exactly. |
| +- **Console:** |
| + - both doors' gates; |
| + - `--from` restore, cap, semaphore and scratch cleanup on every path; |
| + - import begin, upload, stopped and abort in every state; |
| + - hash and length refusal; |
| + - the protocol floor; |
| + - the audit rows. |
| +- **CLI:** `instances create --from`, `instances import` end to end against |
| + a fake console. |
| +- **On the dev box:** |
| + - a `--from` canary of a real instance; |
| + - an import of a stopped snapshot over an addr row; |
| + - an abort after the swap. |
| +- **Cross-compile:** darwin and windows builds stay green. |
| + |
| +## 10. Rollout and #261's schedule |
| + |
| +1. Build both doors now. They refuse on flat boxes, so they are safe to |
| + ship before Tito's boxes are nested. |
| +2. Roll the nested layout to Tito's boxes (#247 steps 2 to 5, Tito's own |
| + approvals). |
| +3. Titogo builds its half (its design §4 and §7: snapshot, import, carry |
| + env, swap, repoint, retire; import mirrors that write through the |
| + instance; controld's `retire`). |
| +4. Convert one canary host, verify, abort it once on purpose, convert it |
| + again. |
| +5. Convert the rest, a few at a time, with Paul's go. |