carlos / platform Public

carlos db push / pull

Adds commands that copy an instance's SQLite database between a local machine and the server. They go through the console API only, so the user needs no box or AWS access.

carlos db pull    --app A [--host H] [PATH] [--backup ID|latest] [--force]
carlos db push    --app A [--host H] PATH [--yes]
carlos db status  --app A [--host H]
carlos db backups --app A [--host H]

Before building this, I checked that no existing CLI command, console route or discussion already covered it.

Design

• Scope: Hibernating, activator-managed exec instances only. They share the class check used for instance moves, which is now factored into one shared gate. A replica can be pulled but not pushed.
• Transport: Files are staged through the console into the control bucket, under apps/<sqid>/<app>/db/<host>/..., with a 256 MiB cap, the same way carlos ship works. - The transfer's state lives on the instance record as a new, optional db field, written with a compare-and-swap (SetRecordDB). - The box's dbPass, which runs beside movesPass, does the actual work.
• Pull: The box restores the litestream replica to a scratch file without taking the lease, checks it with CheckDB and stages it. The CLI downloads the file, then cleans up the staged copy. - On a running instance the copy can be up to one sync interval behind.
• Push, CLI side: The CLI runs integrity_check and VACUUM INTO locally. It asks for confirmation (--yes, or typing the host name), then uploads.
• Push, box side: 1. The route status goes to maintenance, and a fence stops the instance waking. 2. The box waits for the instance to stop, then checkpoints the current database. 3. It takes a backup; the five newest are kept. 4. It deletes the replica's files and replicates the new database as a fresh lineage. 5. It purges the local warm cache, restores the route status and lifts the fence.

The next wake restores the pushed database through the existing restore and CheckDB path. - Every step is resumable. - A failure between the delete and the new replication keeps the route in maintenance and retries, so the instance can't wake with an empty database. - The argument for each crash point is in the header of internal/host/dbpass.go.

• Access: The console routes are closed to every service role.
• Moves: A move and a database transfer each refuse to start while the other is in progress.

Testing

• Litestream: internal/lease/replace_test.go uses the real vendored litestream. It replicates database A, deletes the replica's files, replicates database B, and restores exactly B, including when the sequence is repeated (the resume case).
• Unit tests: - Record transitions: all covered. - Console routes: ownership 404, the confirm header, the file-type and size checks, the replica and move refusals, and the service-principal refusal. - Box worker: pull, push, resuming from each step, a corrupt upload, a crash after the delete, and status and fence restored after a failure. - CLI: host inference, the overwrite refusal, and the non-interactive confirmation refusal.
• Local gate: go build and go vet are clean. In go test ./..., internal/edge and internal/tunnelclient fail in the same way on a clean tree. That is a go1.27 + x/net toolchain issue, not this branch. gofmt -l flags only internal/console/archlegibility_test.go, which is already unformatted on main.

Known limits and follow-ups

• The console holds a whole upload or download in memory; the 256 MiB cap is what bounds it.
• A push whose S3 calls keep failing stays in maintenance until they succeed. There is no way to abort an in-flight push yet.
• No second model has reviewed this yet, so it needs a review before it lands.