carlos / platform Public

spec: move any app or instance to another region, safely (#175)

The move half of Paul's 2026-10-02 goal, composed with the nearest-region
spec (app region) and the #199 replica-ownership spec (generations).

Answers the 2026-10-01/02 production findings: the per-host files
directory and <host>.env are carried in a sealed archive keyed by
incarnation and generation; a target-readiness report (secrets key, config
render, store reach, archive, disk, release, one owner) gates the cut; the
target lands under a durable fence, probes, and commits before it opens;
a failed landing flips region and DB pointer back automatically (M2).
Specifies P3, P5 with signed deletion warrants, app moves with the store
following the app (M3), owner and operator doors (M1), the CLI and
console, invariants with a crash simulation, rollout, five phases, twelve
open questions, draft copy, and fifteen interface points with #199.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude 3 weeks ago e41c4fef67826e2af60e52ce5ded568c6fc9ac28 parent 7edd4b3

1 file changed, +1470 −0

docs/superpowers/specs/2026-10-02-move-any-app-design.md +1470 −0

diff --git a/docs/superpowers/specs/2026-10-02-move-any-app-design.md b/docs/superpowers/specs/2026-10-02-move-any-app-design.md

new file mode 100644

index 000000000..be8336431

--- /dev/null

+++ b/docs/superpowers/specs/2026-10-02-move-any-app-design.md

@@ -0,0 +1,1470 @@

+# Move any app or instance to another region, safely

+**Date:** 2026-10-02

+**Status:** DRAFT for review. Nothing here is built.

+**Discussion:** amadan `carlos/platform` #175 (instance moves). Its last

+four posts are the production findings this spec answers.

+**Goal (Paul, 2026-10-02):** "I want region-default AND the ability to

+easily move any app or instance to a different region."

+**Companion:** `docs/superpowers/specs/2026-10-01-nearest-region-default-design.md`

+on branch `design-nearest-region` (approved). That spec gives every app a

+region at creation. This one moves apps and instances afterwards.

+**Builds on:**

+- #331, `docs/superpowers/specs/2026-08-28-instance-move-design.md`: the

+ two-phase, data-following move. Its never-two-writers cut ordering

+ stands and is extended, not reopened.

+- The moves design record, `docs/superpowers/specs/2026-09-24-instance-moves-for-every-app-design.md`

+ on branch `docs-instance-moves-design` (called "the 09-24 record" below),

+ with its §0 rulings and slices P0 to P9, and

+ `CODEX-RULINGS-INSTANCE-MOVES.md` beside it.

+- On `main` (7edd4b33): P0 (cut fence) and P7 (edge door), merged and

+ live; the move-ready fix (a round promotes at a delta of at most 16 MiB,

+ and copy receipts handle multipart ETags).

+- **Replica ownership (#199)**, `docs/superpowers/specs/2026-10-02-replica-ownership-and-db-push-design.md`

+ on branch `design-replica-ownership` (rev 2 at ef0b7e45, being

+ rewritten; called "the #199 spec" below). Approved as a constraint on

+ this spec by Paul on 2026-10-02. §0.3 says what it changes here, and §26

+ lists every interface point between the two specs, so they can be

+ reviewed together.

+Citations are to this worktree at 7edd4b33 unless they name another

+branch or repo.

+## 0. Rulings in force

+### 0.1 Paul's rulings for this spec (2026-10-02). Not reopened here.

+| id | ruling |

+|---|---|

+| M1 | **Who:** owners move their own apps and instances, self-serve, from the console and the CLI. Operators can do the same for any account. The platform refuses unsafe moves. |

+| M2 | **Downtime:** a short cut window (tens of seconds of 503), an automatic post-cut health check, and **automatic rollback to the source if the target will not start**. Not live migration. |

+| M3 | **Stores move with the app:** an app's object store is copied to the target region and rebound, so residency follows the app (the 09-24 record's P8). |

+### 0.2 The 09-24 record's §0 rulings, as they apply here

+They bind unless M1 to M3 override them. Two are overridden, and in a

+narrow way:

+- **R4 ("rollback is a move back, never an automatic undo") is narrowed by

+ M2.** Before the target has admitted any public traffic, a failed landing

+ is undone automatically by flipping the record back (§7.6). Nothing of

+ value is lost, because no user write reached the target. After the

+ target opens, R4 stands exactly: recovery is a move back, and nothing

+ ever restores an older source copy over newer target data (the 09-24

+ record's M2).

+- **Q5 ("other accounts are operator-only until #171") is superseded by M1

+ for an owner's own apps.** §14 says how the residual #171 risk is

+ bounded: the console's own move ledger, not the record, authorizes every

+ privileged act a move performs.

+Everything else binds as written: R1 to R3 and R5, Q1 to Q4 and Q6 to Q8,

+M1 and M2 of that record (write fence for stores, recovery path for a stuck

+landing), P4 joining the minimum path, and P7-trust.

+### 0.3 The #199 constraint (Paul, 2026-10-02)

+This spec assumes the #199 spec's storage model and is written against it:

+- **Replica data is keyed by incarnation and generation.** A record's

+ replica lives at `<Root>/<host>/<incarnation>/<Gen>/`, and the record's

+ `DB` pointer names the current generation (#199 §2.2, §2.3). The local

+ database carries a `.lineage` file naming the storage key it belongs to

+ (#199 §2.5).

+- **Records that exist when #199 rolls out keep the legacy host-only

+ layout** (`instances/<host>`) until their first v2 event. A record is

+ legacy while it has no `DB` pointer and was created before the

+ deployment's v2 epoch (`control/storage-v2.json`, #199 §2.3).

+- **A move lands its copy as a new generation in the target, and the flip

+ sets `Region` and `DB` in one CAS** (#199 §4). A stale copy from an

+ earlier stay in the target has an older generation and is never read.

+ "Staging wins" (`backend_exec.go:1581-1588`) goes away for v2 records.

+- **The source still writes only `moves/*` in other regions.** No new IAM.

+- **Still undecided in #199:** whether landing copies the staged

+ generation into `instances/`, or the pointer names the staging root

+ (`Root: "moves"`). This spec works with either and recommends one (§26,

+ I-1).

+Where this spec says "generation" it means #199's `Gen`. Where it gives a

+legacy path, that path applies only to records #199 classifies as legacy.

+## 1. What happened in production, and why

+### 1.1 Finding 1: moves never reached `ready` (fixed on main)

+`bdn/ips` `kpmg.bdn.oncarlos.com` sat in `preparing` for 80 minutes. The

+cause was multipart ETags: litestream uploads large LTX files in parts,

+`CopyObject` gives the copy a single-part ETag, and the mirror's ETag

+comparison never matched. Fixed in 2cbf6196 and 167d721b (merged as

+7edd4b33): copy receipts in the source's own bucket at

+`move-receipts/<host>.json` (`internal/activator/backend_exec.go:1557`)

+and a 16 MiB readiness bar (`internal/host/movespass.go:215-235`, applied

+at `:387`). Nothing more is needed for it here.

+### 1.2 Finding 2: a completed move left the instance unable to start for 1h54m

+kpmg moved IE (home) to uk cleanly and then failed every wake on the UK

+box with `instance not healthy within 30s`

+(`internal/activator/backend_exec.go:1443`). Two causes.

+**(a) The per-host data directory does not move.** The move carries the

+litestream replica and nothing else. The cut mirrors

+`instances/<host>/` into the target's `moves/<host>/`

+(`movespass.go:476-487`), purges the local database artifacts

+(`:492`, `PurgeLocalDBArtifacts` at `backend_exec.go:1566-1575`, which

+removes the DB, WAL, SHM and the two litestream meta directories and

+nothing else), and flips the record (`:494`). The app keeps its

+`instance.key` and `report.token` in `/data/<host>/`, beside its database.

+No platform code knows that directory exists: no path in this repo is

+built from it, nothing backs it up, and nothing deletes it. On the UK box

+the app found no key, minted a new one, and could not read data sealed

+under the old one.

+**(b) `<host>.config.env` was never rendered on the target.** Exec

+config is rendered by the agent's in-process `SyncExec`

+(`internal/appconfig/syncexec.go`, `ConfigEnvPath` at `:71-76`), which runs

+only when the box holds `CARLOS_SECRETS_KEY`: both config loops log

+"disabled" and return without it (`internal/host/agent.go:881-886`,

+`:1279`), and the fast converge skips the config leg when the key is absent

+(`agent.go:1812`). The spawn treats a missing config file as an empty one

+(`backend_exec.go:1146-1149`), so the app started with no store binding

+and logged "build app: File storage is unavailable." The UK box had

+`<host>.env` (the activator mints it at first wake,

+`internal/activator/prepare.go:49`) but no `<host>.config.env`.

+**Which of three causes stopped the render is not yet confirmed.** In

+order of likelihood: the UK box has no `CARLOS_SECRETS_KEY` in

+`/etc/carlos/host.env` (the loops are then inert, see §6); it holds a

+different key, so every sealed value fails to open; or `SyncExec` ran and

+skipped the route. The UK agent's journal answers it in one line (the

+"config sync loop disabled" message at `agent.go:884`). Phase 1's first

+task reads it. The design below does not depend on the answer: the

+target-readiness report (§7.3) catches all three before anything is

+stopped.

+Recovery was a hand edit of the record's region back to home plus a serial

+bump. It worked because the source still had the key in `/data/<host>/`

+and its own bucket still had the replica. Had anything deleted

+`/data/<host>/` on the source, the data would have been unrecoverable.

+### 1.3 Finding 3: the rehearsal proved nothing

+`bdf/hello` moved IE to uk in about 6 minutes with about 50 s of 503. It

+has no key file, no store, no config and no scheduled jobs, so it

+exercised none of the state that failed. §20.4 specifies a rehearsal app

+shaped like ips.

+### 1.4 Finding 4: SSM Parameter Store is regional

+The carloku-infrastructure root says so plainly: "SSM Parameter Store is

+regional: this host reads /carlos/* in ITS OWN region, so every parameter

+it needs (release key, app secrets) must be created in eu-west-2. Nothing

+replicates them from eu-west-1." (`region-uk.tf:61-64`; the same note is in

+`region-us.tf` and `region-au.tf`). §6 traces what that means for an

+instance.

+## 2. Goals and non-goals

+Goals:

+- **Any eligible instance moves safely.** Every piece of per-host state is

+ carried or provably reproduced on the target, checked before anything

+ is stopped, and checked again before the target serves.

+- **A target that cannot run the instance gives it back.** The source

+ serves again automatically, with the data it had at the cut.

+- **An app moves as a unit.** Its region changes, then its instances move

+ one at a time, then its object store follows (M3).

+- **Owners do it themselves** from the console and `carlos move` (M1).

+- **The source region is emptied** after a hold period (P5), so a move

+ delivers the residency it was bought for.

+Non-goals:

+- Live migration, zero-downtime cuts, or pre-warming the target (§23 Q8).

+- Moving pooled, fleet-placed, addr-backed, unit-backed or non-hibernating

+ instances, replicas, or homes with replicas (§13).

+- The freeze/thaw/verify hooks (the 09-24 record's P1). They make a cut

+ graceful; this spec makes it safe without them. They slot in at Phase 5.

+- Adoption of externally run instances (P6) and the isolated layout pass

+ (P9). They stay on the 09-24 record. §8.5 records one requirement P9

+ must now meet.

+- Latency routing for moved instances. Phase L of the nearest-region spec

+ derives it from the record's region, which a move already updates.

+## 3. Words

+- **Instance move:** one record's `region` changes. The app's region does

+ not.

+- **App move:** the app's region changes (the console `apps.region`

+ column, nearest-region spec §5.1), then each of its instances that lives

+ in the old region moves, then the app's store follows.

+- **Source / target:** the box serving the instance before the move, and

+ the box that will serve it after. "Box" means the one box whose registry

+ owns the route (`OwnsRoute`, `movespass.go:180-185`).

+- **The flip:** the single record write that sets `region` to the target.

+- **Landing:** the interval after the flip during which the target

+ restores, checks and probes the instance with no public traffic.

+- **Open:** the target admits public traffic (route status `ready`).

+- **Authoritative copy:** the one copy of the instance's data that a

+ later restore must reproduce. §4 defines where it is in every state.

+- **Hold:** the period after a successful move during which the source

+ keeps its copy (§10).

+- **Incarnation, generation, storage key, epoch:** #199's terms (§2.1 to

+ §2.3 there). The **source generation** is the one the record points at

+ when the move is requested (`move.from_db`). The **move generation** is

+ the new one the move allocates for the target (`move.to_gen`).

+## 4. Invariants

+Review rounds on earlier move work kept finding new interleavings of one

+fault. So the properties are stated once, here, and §20.2 tests them by

+randomized simulation across crashes, not by example.

+- **INV-1 One writer.** At every instant at most one box runs a process

+ for the host. While the target runs one during landing, the source's

+ route is fenced and its child is proven stopped.

+- **INV-2 No open before commit.** The target sets its route to `ready`

+ only after its compare-and-set from `landing` to at-rest has succeeded.

+ A box that loses that CAS never opens.

+- **INV-3 No lost acknowledged write.** Any write the instance acknowledged

+ to a client (a request admitted by an edge, or a tick) is present in the

+ authoritative copy at every terminal state: done, aborted, rolled back,

+ or stuck.

+- **INV-4 Authority.** Before the flip the authoritative copy is the

+ source (its local DB while running, the source generation once

+ stopped). From the flip until the target's commit it is the source

+ generation, which is byte-identical to the final staged copy. After the

+ commit it is the move generation. A rollback restores from the source

+ generation and nothing else.

+- **INV-4b One pointer.** `region` and the `DB` pointer change only

+ together, in one CAS: at the flip (to the target and the move

+ generation) and at a rollback (back to the source and the source

+ generation). No restore on any box reads a location the record does not

+ point at, so an older generation from an earlier stay is never read.

+- **INV-5 Files continuity.** Every start of the instance, on any box,

+ sees the files and `<host>.env` of the authoritative state. No start

+ ever runs where a carried file should exist and does not.

+- **INV-6 Config presence.** No start during landing runs without the

+ instance's config rendered on that box, whenever the app has a config

+ bundle.

+- **INV-7 Termination.** Every move reaches `done`, `aborted`,

+ `rolled_back` or `stuck` in bounded time, provided crashed actors

+ eventually restart. `stuck` happens only when the source also fails to

+ start after a rollback.

+- **INV-8 Residency.** Nothing deletes source state before `done`. After

+ the hold, nothing of the instance remains in the source region except

+ what §10 lists. A deletion never removes a lineage that is live at the

+ moment it fires.

+## 5. Every piece of per-host state, and what happens to it

+"Per-host" means keyed by the instance's hostname or its instance id

+(`provision.InstanceID`, `internal/provision/provision.go:1259`). The

+flat layout keeps files under `<DataDir>` (`/data`); the nested

+(isolated-exec) layout keeps them under `InstanceDir` =

+`<DataDir>/instances/<iid>/` (`provision.go:1270`;

+`internal/activator/paths.go:33-54`).

+|---|---|---|---|---|---|

+| S1 | SQLite DB, WAL, SHM | `<DataDir>/<host>.db` (`provision.go:1219`) or `InstanceDir/<host>.db` | app | carried as the replica; source copy purged at cut (`movespass.go:492`) | unchanged |

+| S3 | Replica lineage | legacy: `instances/<host>` (`activator.go:138`); v2: `<Root>/<host>/<incarnation>/<Gen>/` in the box's own data bucket (#199 §2.3) | activator | mirrored to target staging `moves/<host>/` (`backend_exec.go:1549`) and promoted over the target's `instances/<host>` (`:1580-1606`); source copy kept forever | mirrored into the **move generation** `moves/<host>/<incarnation>/<to_gen>/` in the target bucket; the flip points the record at it (#199 §4); the source generation is deleted after the hold (§10) |

+| S4b | Local lineage stamp | `<dbPath>.lineage` (#199 §2.5) | activator | n/a | removed first by the cut's purge (#199 §5.6); written on the target after its first sync |

+| S5 | **Instance files** | flat: `<DataDir>/<host>/` (app-created: `instance.key`, `report.token`); nested: files the app writes in `InstanceDir` that are not platform-owned | app | **not carried, not backed up, never deleted** | **carried** in the sealed files archive (§8) |

+| S6 | **`<host>.env`** | `<DataDir>/<host>.env` (`paths.go:50`) | activator mints `CARLOS_ADMIN_TOKEN` (`admintoken.go:115`, `prepare.go:49`); operators add box-local overrides, which win over the bucket (README "App config & secrets") | **not carried**: target mints a new token, overrides are lost | **carried** in the files archive |

+| S7 | **`<host>.config.env`** | `ConfigEnvPath` (`syncexec.go:71-76`) | `SyncExec`, only with `CARLOS_SECRETS_KEY` (`agent.go:881-886`, `:1812`) | rendered only if the target holds the key; absence is silently accepted at spawn (`backend_exec.go:1146-1149`) | reproduced from the bucket; **proven** on the target before the cut (§7.3) and again before the probe (§7.5) |

+| S11 | Timers: declared schedules | `timers` table, `kind=schedule` (`internal/registry/registry.go:565-579`) | converger from `schedules.json` (`internal/schedule/converge.go:17-20`) | rows re-created on the target with `next_at`/`last_at`/`attempts` reset | **carried** with their state; the converger keeps an imported row whose spec and path match (`converge.go:223-241`) |

+| S12 | Timers: once-timers | `timers` table, `kind=once` (`internal/schedule/spec.go:19`), registered by the app over its control socket (`control.go:41`) | app | **lost** (09-24 G15) | **carried** |

+| S15 | Custom domains and certs | aliases derived per box from claims (`internal/domains/aliases.go`); certificate cache in the deployment bucket under `control/certs/` (`internal/edge/certsweep.go:4`) | domains pass, edge | reproduced; 404 between the source's retire and the target's open | reproduced; the 404 becomes a 503 with P2 (Phase 2) |

+**S5 needs a definition the platform can act on.** No code in this repo

+names `/data/<host>/`. It is the app's own convention: a directory named

+for the host beside its database. The design makes it a platform contract:

+- **Flat layout:** the instance files are the directory

+ `strings.TrimSuffix(dbPath, ".db")`, which is `<DataDir>/<host>/` for

+ every row whose `DBPath` is the default.

+- **Nested layout:** every regular file and directory under `InstanceDir`

+ except the platform's own names (S1, S2, S4b, S6 is carried separately,

+ S7, S17, S19, and #199's `.incarnation` marker).

+- Anything an app writes elsewhere under `/data` is not carried.

+ `docs/site` says so, next to the existing rule that the app must read

+ its store coordinates from env.

+**The files belong to the incarnation, and a move hands them to a new

+generation.** The #199 spec gives the nested working directory an

+`.incarnation` marker and moves a mismatched directory aside to

+`<DataDir>/.retired/<old-incarnation>/` (#199 §2.5b). A push replaces only

+the database, so `instance.key` survives a push. A move is different: it

+creates a new generation **in another box**, and the files have to arrive

+with it. So the files are keyed the same way the replica is, in both

+places they live:

+- **In the bucket**, the archive sits beside the move generation, at

+ `moves/_files/<host>/<incarnation>/<to_gen>.{archive,json}` (§8). A

+ stale archive from an earlier stay has another generation in its key and

+ is never read.

+- **On a box**, the directory carries `.incarnation` (#199) and a second

+ marker this spec adds, `.files-gen`, naming the generation whose landing

+ wrote it. A landing never trusts a directory it did not just write: it

+ moves whatever is there aside and extracts fresh (§7.5, L2). A push

+ leaves `.files-gen` alone, because the files did not change.

+- **Flat rows**, which #199 cannot attribute (§2.5b there), become

+ attributable for this one directory, because this spec makes

+ `<DataDir>/<host>/` a platform-defined path. #199's `.incarnation` check

+ should extend to it (§26, I-6).

+## 6. SSM, secrets, and what a target box needs

+The platform's Go code reads no SSM parameter at runtime. There is no SSM

+client in this repo. The region-host module only grants a box

+`ssm:GetParameter` on one ARN pattern (`infra/modules/region-host/main.tf:179-180`)

+and denies every other parameter read (`:435-450`). So SSM reaches an

+instance in exactly two ways, and only one of them matters to a move:

+1. **Box provisioning.** `/etc/carlos/host.env` is staged per box, and its

+ values come from the deployment's secret store, which on carloku is SSM

+ in the box's own region. The value that matters is

+ **`CARLOS_SECRETS_KEY`**: it decrypts every sealed config value, and the

+ README says it "belongs in exactly two places, your secret manager and

+ every host's `/etc/carlos/host.env`". A box without it renders no exec

+ config at all (§1.2b).

+2. **Apps that read SSM themselves.** Today these are unit-backed adopted

+ apps whose deployment scripts read `/carlos/kass/*` and

+ `/carlos/keymail/*` (carloku `README.md:16-18`, `scripts/copy-kass-secrets.sh`).

+ Unit-backed routes never move (`internal/console/apiinstancemove.go:53-60`).

+An exec instance's own secrets come from its config bundle in the

+deployment bucket, which lives at home (Q4) and which every box can read.

+**So nothing per-app or per-instance needs to be replicated between SSM

+regions. What has to be true is that every target box holds the

+deployment's secrets key.** That is a box property, so the design checks

+it as one:

+- Each box publishes, in its edges file, the fingerprint of the public half

+ of its `CARLOS_SECRETS_KEY` (`secrets_key_fp`, additive and omitempty,

+ beside `placement` and `data_bucket` at `internal/host/edgesfile.go:603-605`),

+ or `""` when it has none.

+- The console publishes its own `CARLOS_SECRETS_PUBKEY` fingerprint.

+- The move door refuses a target whose fingerprint is absent or different

+ (§7.3, check T2). That one refusal would have stopped the kpmg move

+ before the cut.

+Operationally, the roll recipe gains one line: staging a new region means

+creating its SSM copy of the secrets key, and the edges file proves it

+landed. A follow-up for carloku-infrastructure (not this repo) is a small

+tofu resource that copies `/carlos/secrets-key`, or whatever the

+deployment names it, into each region from one source of truth.

+## 7. The instance move

+### 7.1 States

+The record keeps #331's in-flight object (`provision.MoveState`,

+`provision.go:907-917`) and its CAS (`SetRecordMove`, `:949`). New states

+and fields are additive. A record written by today's console still parses.

+```

+preparing ─► ready ─► cutting ─► landing ─► (at rest in target; log: done)

+ │ │ │ │

+ │ │ │ └─► rolling_back ─► (at rest in source; log: rolled_back)

+ │ │ │ └─► stuck (source failed too; operator)

+ └──────────┴─────────┴─► aborting ─► (at rest in source; log: aborted)

+ (cutting only before the source stops anything: cut_refused ─► ready)

+```

+|---|---|---|---|

+New fields on `MoveState`, all omitempty:

+```json

+"move": {

+ "id": "mv_01J9…", // console-minted; the generation nonce, the log, the ledger

+ "from": "", // record region before the move ("" = home)

+ "from_db": {"gen": "g3-…", "root": "instances"}, // the source generation; null = legacy

+ "to_gen": "g4-mv_01J9…", // the move generation (#199 §4)

+ "to": "uk",

+ "state": "landing",

+ "cut": "auto", // "" = manual cut door (today's behaviour)

+ "requested_at": "…", "ready_at": "…", "updated_at": "…",

+ "cut_at": "…", // source stamps when it sets the route "moving"

+ "flipped_at": "…",

+ "landing_deadline": "…", // flipped_at + 10 min

+ "files": {"digest": "sha256:…", "bytes": 18234, "entries": 4},

+ "reason": "UK could not start the instance: …" // rollback, refusal or stuck

+```

+`MoveRequest` (`provision.go:966-983`) records `from`, `from_db` (the

+record's `DB` at that moment, or null for a legacy record) and allocates

+`to_gen = g<seq+1>-<move id>`, where `seq` is the source generation's

+sequence (0 for a legacy or pointerless record, so a legacy record's first

+move lands at `g1-<move id>`, #199 §4). The move id is the nonce, so no two

+moves, and no move and push, can share a location.

+`MoveComplete` (`provision.go:1056-1065`) changes in two ways. In one CAS

+it writes `region = to`, `DB = {Gen: to_gen, Root: <I-1>}` and

+`state = landing`, instead of clearing `move`. The target clears `move` at

+its commit. Region and generation flip together in exactly one write

+(#199 §4), and the source has nothing to do after it. This is the 09-24

+record's §6.1 `landing`, with #199's pointer added.

+Consoles roll first (§21). P0's allowlist in `MoveAbort`

+(`provision.go:1027-1035`) already refuses any state it does not know, so a

+console from v0.22 cannot abort a `landing`, `rolling_back` or `stuck`

+record.

+### 7.2 Preparing

+The replica rounds are main's delta rounds of `lease.MirrorPrefix`

+(`movespass.go:372-394`) with #199's keys: from the source storage key

+(`StorageKey(rec)`, #199 §2.3) into the target's

+`moves/<host>/<incarnation>/<to_gen>/`, with receipts at the source-keyed

+receipt path (S4). Two additions run in the same pass:

+1. **Files-archive rounds (source).** Each round, the source builds the

+ instance files archive (§8) from the live directory, and uploads it only

+ when its digest has changed since the last upload. It goes to the

+ target's data bucket at `moves/_files/<host>/<incarnation>/<to_gen>.archive`

+ with a manifest at `….json`. The `_` makes the key impossible to

+ collide with a host (no valid host contains one, `provision.go:1252-1258`),

+ and it sits inside the existing push-only `moves/*` grant

+ (`infra/modules/region-host/main.tf:217-227`). No new IAM.

+ This archive is a rehearsal copy taken from a running instance, so it

+ may be inconsistent. It proves the path and the key, not the content.

+ The final archive is taken at the cut.

+2. **Readiness reports (target).** §7.3.

+`ready` keeps today's rule (delta at most the bar) and gains one: the

+newest target report for this move id passes and is under 10 minutes old.

+### 7.3 Target readiness (the pre-flip check)

+The target box gains a move-worker role. Its pass already lists every

+record (`movespass.go:242`). For each record whose `move.to` is this box's

+placement and whose state is `preparing` or `ready`, it runs the checks

+below and writes the result to the deployment bucket at

+`moves/<acct>/<app>/<host>/<move-id>/target.json`. Every check runs while

+the source keeps serving.

+| check | passes when | catches |

+|---|---|---|

+| T1 capability | this box runs a build that understands `landing` and the files archive, and its edges file publishes `moves_schema >= 2` | an old target minting a `ready` route with no checks: today's failure mode |

+| T2 secrets key | this box holds `CARLOS_SECRETS_KEY` and its public fingerprint equals the console's | §1.2b, every variant |

+| T3 config render | a dry run of `SyncExec`'s resolution for this host's (account, app, environment) plus the instance overlay decrypts every sealed value; the report lists the variable **names** and the bundle etags, never a value | a bundle the box cannot open, a missing environment |

+| T4 store reach | for every store bound in that render (`<PREFIX>_BUCKET`, `_REGION`, `_ENDPOINT`, `_ACCESS_KEY`, `_SECRET_KEY`, `internal/objstore/objstore.go:214-218`), a `HeadBucket` with the bound credential from this box succeeds | "File storage is unavailable" |

+| T5 files archive | the newest staged archive opens with this box's key and its digest matches its manifest | the key path for S5 and S6 |

+| T6 disk | free space on the data volume (`readDiskFrac`, `internal/host/probes.go:209`) covers twice the replica size, plus the archive, plus 2 GiB, and stays under 85% full afterwards | the 91% Ireland disk of 2026-10-01 |

+| T7 release | the instance's channel head materializes on this box (the check a wake would make; "no adoptable head", `internal/activator/logsink.go:64`) | a channel with nothing promoted for this box's architecture |

+| T8 one owner | exactly one hosting box publishes the target placement | a second same-region box minting the route (the `OwnsRoute` reasoning at `movespass.go:180-185`, from the other side) |

+The console's cut door refuses with the failing check's sentence (409)

+unless the newest report passes. The source re-reads the report as the

+first step of every cut pass and refuses before it stops anything

+(`cutting` back to `ready`, with `reason`), because records have other

+writers. An auto-cut (§16) is the source taking this same step itself.

+### 7.4 The cut (source)

+#331's ordering (`movespass.go:43-51`), with three steps added (marked

+NEW):

+1. NEW: re-read the target report; refuse the cut if it no longer passes.

+2. Route status `moving`; stamp `cut_at` once (`:418`).

+3. Wake fence (`:424-431`); wait until `Quiesced` (`:432`).

+4. Wait out a layout journal (`:447`).

+5. Final checkpoint into the source generation (`:470`).

+6. Byte-final mirror round into the move generation (`:476-490`).

+7. NEW: **final files archive**: build it from the stopped instance's

+ directory, upload it, record its digest. Export the host's `timers` rows

+ into it (§8.1).

+8. Local DB purge (`:492`), with #199's error-returning purge, which

+ removes `.lineage` first and fails the pass on any error other than

+ not-exist (#199 §5.6). **The instance files and `<host>.env` are not

+ touched.** They are the rollback's material until the hold ends.

+9. NEW in content: the flip writes `region = to`, `DB = {to_gen, Root}`,

+ `state = landing`, `flipped_at`, `landing_deadline` and `files` in one

+ CAS, guarded by the move id and `from_db`.

+Every step stays idempotent, and a box that dies mid-cut resumes at the

+top. The archive build is deterministic (§8.1), so a resumed cut re-uploads

+a byte-identical object.

+### 7.5 Landing (target)

+The target's reconcile mints the route when the record's region names it,

+as today, with #199's `incarnation`, `storage_key` and `storage_gen`

+columns rendered from the record (#199 §2.4), and with one change: a

+record in `landing` is minted with status `landing` instead of `ready`

+(`reconcile.go:374`). That status is the durable fence. It survives an agent restart, which the in-memory activator

+fence does not (`movespass.go:294-300`):

+- the edge serves only `ready` rows (`internal/edge/edge.go:595`), so no

+ visitor can wake it;

+- ticks refuse any non-`ready` row (`internal/activator/activator.go:2785-2789`).

+The target's move worker then runs, in order, each step idempotent:

+| step | action | on failure |

+|---|---|---|

+| L1 | fetch the archive at `moves/_files/<host>/<incarnation>/<to_gen>`; verify it against `move.files.digest`; open it | roll back: `files archive missing or damaged` |

+| L2 | move any existing files directory aside to `<DataDir>/.retired/<incarnation>-<old .files-gen or "unknown">/` (#199 §2.5b's convention, never a delete); extract fresh (§8.3); write `.incarnation` and `.files-gen = to_gen`; write `<host>.env`; import the timers | roll back |

+| L3 | run `SyncExec` for this host; require a config ack from this box for this host written after `flipped_at` (`configack.go:116`), whenever the app has a bundle; require every name in T3's list to be in the file | roll back: `configuration did not render on <target>` |

+| L4 | repeat T4 with the rendered file | roll back: `<target> cannot reach the app's storage` |

+| L5 | **probe wake**: a new activator entry, `ProbeWake(host, moveID)`, legal only on a `landing` row. It restores cold from the row's `storage_key`, which the record's pointer names (#199 §2.4; no `.lineage` exists yet, so no warm file can be used), runs `CheckDB` (`backend_exec.go:313`), spawns, and requires the existing health answer (an HTTP response under 500 on the socket within 30 s, `backend_exec.go:36`, `:1426-1444`). Then it waits a 20 s settle window and asks again. Up to three attempts. Whether the restore reads the staging root directly or a copy under `instances/` is I-1 (§26) | roll back: the member-safe wake reason (`logsink.go:49`) |

+| L6 | **commit**: CAS `landing` → at rest (clear `move`), conditional on the record still saying `landing` with this move id | lost CAS: never open; follow the record (it was rolled back) |

+| L7 | **open**: route status `ready`; bump the serial; log `done` with the timings | retried every pass until true |

+INV-2 lives in L6 before L7. A target that crashes between them finds a

+record at rest in its region and a `landing` row on restart, and finishes

+L7. Nothing else writes route status, so this repair is the worker's job

+(#175's P0 notes: "nothing writes a `moving` row back to `ready`").

+**What visitors see.** From the source's `moving` until L7, the host has no

+`ready` row anywhere. A host under a platform apps domain gets the 503

+pending page (#174, `internal/edge/pending.go`). A custom domain gets 404

+until P2's holding route lands (Phase 2). That is the window M2 calls

+"tens of seconds of 503". §20.4 measures it on an ips-shaped instance.

+### 7.6 Automatic rollback

+A failed landing check, or a passed `landing_deadline`, rolls the move

+back. This is the M2 promise and it needs no staging lane, so it works when

+the source is the home region (kpmg's case) without the home placement of

+§9.

+**Who writes it.**

+- The target, on any L1 to L5 failure: first stop its child and purge its

+ local DB (error-returning, `.lineage` first) and config, and move its

+ freshly extracted files aside; then CAS `landing` → `rolling_back`,

+ setting `region = move.from`, `DB = move.from_db` and `reason`.

+- The source, when `now > landing_deadline` and the record still says

+ `landing`: the same CAS. The target may be dead or wedged.

+Both are CASes from `landing`, guarded by the move id and

+`DB.Gen == to_gen`, so exactly one of commit (L6) and rollback wins. A

+target that loses to the source's rollback never opened (INV-2), so it

+holds no acknowledged write (INV-3). Region and pointer go back together

+(INV-4b): the rollback is the flip, reversed, in one write. This is why

+rollback needs neither a staging lane into the source nor any copy.

+**What the source does** in `rolling_back` (its placement matches the

+record region again):

+1. If its `moving` row still exists: set it `ready` and lift its fence.

+ Otherwise reconcile re-mints the route. Either way the row ends up

+ `ready`, with the source storage key. If the source row had been

+ retired and re-minted, #199's retire-on-identity-change (§2.6 there)

+ does not fire, because the incarnation and storage key are the ones the

+ row was minted with.

+2. **Files check.** The files directory and `<host>.env` are where the cut

+ left them (§7.4 step 8). The source re-hashes them against the

+ archive's per-file manifest (§8.2), which it wrote at the cut. A

+ mismatch means something changed the directory after the cut: go to

+ `stuck`, never start (INV-5). The config file is re-rendered by the

+ same pass.

+3. Probe wake. The restore comes from the source generation, which equals

+ the final staged copy (INV-4). The cut removed the local database and

+ its `.lineage`, so the warm-cache path cannot win.

+4. Probe passes: CAS `rolling_back` → at rest; log `rolled_back` with the

+ target's reason; page as a warning.

+5. Probe fails three times: CAS → `stuck`; page as an incident.

+The source admits traffic as soon as its row is `ready`. That is safe: the

+source copy is the authoritative one in this state.

+**What the target leaves** once the record has left its scope: reconcile

+retires the `landing` row (its incarnation is unchanged but the record no

+longer names this region). The move generation

+`moves/<host>/<incarnation>/<to_gen>/` and its archive are **unreferenced**:

+nothing points at them, and no later move can reuse the key, because the

+move id is in it. They are garbage, not a hazard (#199 §4), so residency

+cleanup deletes them on the next pass rather than as part of the

+rollback. A denied delete (`movespass.go:509-512`) is logged as residue.

+The target's moved-aside files directory goes the same way.

+**`stuck`** keeps `region = from` and the full `move` object as evidence.

+New moves are refused. An operator resolves it with

+`carlos move resolve --host <h>` after fixing the box (§16). The verb

+re-runs the source probe, and on success clears the move. It never touches

+data. This is the 09-24 record's M2 recovery path.

+**Side effects.** An app that does work at boot (sends mail, calls a

+webhook) will have done it on the target during L5, and will do it again on

+the source after a rollback. `docs/site` says so. Apps that care can read

+`CARLOS_MOVE_PROBE=1`, which the probe wake sets, and skip boot-time side

+effects.

+### 7.7 Abort

+Legal in `preparing`, `ready`, and in `cutting` before the source has set

+the route `moving` (`cut_at` unset). The source's abort arm

+(`movespass.go:396-413`) deletes the move generation and its archive

+before it clears the move. Under v2 that delete is residency hygiene, not

+safety (an unreferenced generation is never read), so a failed delete no

+longer has to hold the abort open; it is logged as residue. For a legacy

+record the abort keeps today's two-phase rule, because legacy staging

+still wins promotion (§7.9). Abort after `cut_at` is refused. Past that point the move

+either completes or rolls back on its own, and both are faster than an

+abort would be.

+The 09-24 record's "abort until the purge" (cutting before the purge) is

+not adopted. Automatic rollback covers the same ground with one mechanism

+instead of two.

+### 7.8 Crash at any step

+| actor crashes in | on restart |

+|---|---|

+| source, `preparing`/`ready` | next round resumes; receipts make it cheap |

+| source, `cutting` before the flip | resumes at step 1 (#331); re-uploads a byte-identical archive |

+| source, after the flip | nothing to do until `rolling_back` or the deadline |

+| target, before L6 | re-runs L1 onwards; extraction and import are idempotent; the probe counts attempts per process, and the deadline bounds the total |

+| target, between L6 and L7 | sees record at rest plus a `landing` row; does L7 |

+| target, during its rollback | record already `rolling_back`: finishes cleanup; or not yet written: re-runs the failed check, fails again, writes it |

+| source, during `rolling_back` | re-runs §7.6 steps 1 to 5 |

+| console, any state | stateless for instance moves; the ledger row is re-read |

+| both boxes dead | deadline passes with no writer; the console's move sweep pages at deadline + 5 min, shows `landing (overdue)`, and offers `carlos move resolve` |

+### 7.9 Legacy records

+A record is legacy under #199 §2.3 (no `DB` pointer, created before the v2

+epoch). Two cases.

+**After the epoch exists**, a move is the record's first v2 event (#199

+§4). The source reads its legacy location, `instances/<host>`, and keeps

+the legacy receipt key, `move-receipts/<host>.json`. Everything it writes

+in the target is v2: the move generation is `g1-<move id>`, staged at

+`moves/<host>/<incarnation>/g1-<move id>/`, with the archive beside it.

+The flip writes the record's first `DB` pointer. `from_db` is null, and a

+rollback writes `DB = null` back, which #199 reads as the legacy location

+again, because CreatedAt is still before the epoch. **The incarnation of a

+legacy record whose CreatedAt is zero** is whatever

+`provision.Incarnation` returns for it, as long as source and target

+compute it identically. §26 I-5 asks the #199 spec to pin that.

+**Before the epoch exists**, every record is legacy and #199 writes no

+pointer. The move then keeps main's storage flow exactly: staging at

+`moves/<host>/`, the target promotes it over `instances/<host>` on first

+wake ("staging wins", `backend_exec.go:1581-1588`), and receipts at

+`move-receipts/<host>.json`. Everything else in this spec still applies:

+the archive (at `moves/_files/<host>/<move id>.*`), readiness, `landing`,

+the probe, and rollback. Rollback is still a region flip, since the source

+location never changed. What the legacy flow cannot do is make the

+target's promoted copy unreferenced: after a rollback the target bucket

+holds a full `instances/<host>` lineage that a later restore on that box

+would trust. So, pre-epoch, the target deletes it during the rollback,

+before writing `rolling_back`. If the delete is denied, the rollback goes

+ahead anyway (the instance must not stay offline over cleanup), and the

+residue is logged and shown to operators as a hazard, not as garbage. A

+later move to that box is still safe, because its fresh staging wins

+promotion. What is not safe is anyone pointing the record at that region

+by hand. §23 Q11 asks whether to support this flow at all, or to refuse

+moves until the epoch exists.

+## 8. The instance files archive

+### 8.1 Format

+- A tar stream of the instance files (§5, S5) plus `<host>.env` (S6) plus

+ `timers.json` (S11, S12), in sorted path order, with mtimes and owners

+ zeroed and modes kept. The output is byte-deterministic for one input.

+- Only regular files and directories. A symlink, device, FIFO or socket in

+ the directory refuses the move with its name. The nested layout already

+ refuses special files on tenant ground (34beb1b3), and the archive holds

+ the same line on both layouts.

+- Size cap: **256 MiB** before compression (§23 Q2). Larger refuses the

+ move and points at object storage.

+- zstd, then sealed.

+### 8.2 Sealing

+The archive carries the very key that protects the app's data, so it must

+not sit in a bucket beside that data in the clear. It is sealed to the

+deployment's config keypair, the same one config bundles are sealed to:

+- a fresh 256-bit data key; AES-256-GCM over 1 MiB chunks, each chunk's

+ nonce derived from its index, with the final chunk flagged so the stream

+ cannot be truncated;

+- the data key sealed with ephemeral X25519 to the public half of

+ `CARLOS_SECRETS_KEY` (which `appconfig.ParsePrivateKey` already parses),

+ exactly as a sealed config value is;

+- the manifest (the `.json` beside the archive) holds the sealed data key,

+ the sha256 of the sealed object (the `files.digest` in the record), the

+ plaintext entry list (paths, sizes, sha256 of each file), the move id,

+ the incarnation and `to_gen`. The target refuses a manifest whose

+ incarnation or generation differs from the record's.

+- The source keeps its own copy of the manifest beside the receipts

+ (`move-receipts/<host>/<incarnation>/<from_gen>.files.json`), because it

+ cannot read the target's bucket and the rollback's files check (§7.6

+ step 2) needs it.

+Only a box holding `CARLOS_SECRETS_KEY` can open it. T2 proves the target

+is one.

+### 8.3 Extraction on the target

+Into the path §5 defines for the target row's layout, after moving any

+existing directory aside (§7.5, L2). The target row may be flat when the

+source was nested, or the reverse. The two markers, `.incarnation` and

+`.files-gen`, are written last, by atomic rename, so a crash mid-extraction

+leaves a directory that the next pass recognises as not yet landed and

+replaces. For a nested row,

+writes go through `instancefs` (no symlink follow), as every other write on

+tenant ground does. Every extracted file is checked against the manifest's

+per-file hash. `<host>.env` is written 0600. Timers are upserted with their

+saved `next_at`, `due_at`, `last_*` and `attempts`. Once-timers that came

+due during the cut fire on the first tick after open.

+### 8.4 What it does not carry

+Any path outside §5's definition, and process state. An app that keeps

+files elsewhere under `/data` loses them on a move. `docs/site` says so

+plainly.

+### 8.5 A requirement for P9

+The isolated-exec design moves `<DataDir>/<host>.*` into the instance

+directory (isolated-exec design §2 item 1). That glob does not match the

+directory `<DataDir>/<host>/`, and the unit's `TemporaryFileSystem=/data:ro`

+hides everything outside the instance directory (§4, "load-bearing"

+notes). So the layout pass, as designed, would strand every flat app's

+`instance.key` the same way this move did. P9 must carry `<DataDir>/<host>/`

+into `InstanceDir`, and stamp it with #199's `.incarnation` when it does.

+This is recorded here because it is the same fault, and is to be filed on

+#126.

+## 9. Moves to and from home (P3)

+Automatic rollback (§7.6) does not need this section. Two things do: a

+deliberate move to home (moving back after a successful move away, or an

+app whose region is home), and any move whose source is home and whose

+target needs to stage into home (never; the source stages into the target).

+So P3 is about home as a **target**.

+Today a move to home is refused (`apiinstancemove.go:203-209`), because the

+home box publishes no placement: it runs with `CARLOS_REGION` unset, and

+records store `""` for home. `moveTargets` drops any edges file without a

+placement (`movespass.go:345`), and the home's children receive no

+`CARLOS_REGION` (`backend_exec.go:989-993`).

+The fix is the 09-24 record's §6.6, which R4 ruled on:

+1. **`CARLOS_HOME_PLACEMENT`** on the home box: the console's

+ `CONSOLE_HOME_REGION` name (`ie` on carloku). The box publishes it as

+ `placement` with `"home": true` in its edges file. Its own

+ `CARLOS_REGION` stays unset, so record scope (`""`) and every reconcile

+ that compares by equality are unchanged.

+2. **`moveTargets`** maps `move.to = ""` to the edges file marked home.

+ **`runMoveRound`** looks up `targets[""]` through that map.

+3. **The target's role** (§7.3 and §7.5) matches `move.to == ""` on the box

+ marked home.

+4. **Home children** get `CARLOS_REGION = CARLOS_HOME_PLACEMENT`, so an app

+ sees one variable in every region. This is a behaviour change for every

+ app at home, so it ships behind `CARLOS_HOME_REGION_ENV=on` on the home

+ box, and the roll notes say what it changes.

+5. **The door** (`apiinstancemove.go:203-209`) accepts home once some edges

+ file is marked home with staging coordinates. Until then its refusal

+ says a move to home is not available yet.

+6. **Infra:** every sibling's `sibling_staging_arns` lists the home data

+ bucket. That is production IAM and needs scoped approval per deployment.

+7. **Under #199**, a move to home is an ordinary v2 move: a new generation

+ staged in the home bucket and pointed at by the flip. A boomerang

+ (ie→uk→ie) lands in a fresh `g<n>-<move id>` and never touches the

+ generation the instance left behind in ie, so the 09-24 record's

+ staging-wins reasoning for this case is no longer needed. Until that

+ earlier generation's hold ends, the home bucket holds both; P5 deletes

+ the old one on schedule.

+**Rollback and P3 are separate.** Automatic rollback (§7.6) restores the

+source generation where it already is, in the source's own bucket, and

+works today for a move out of home. P3 is needed only to move **into**

+home deliberately: `carlos move back` after a successful move away, or an

+app move whose target is home.

+## 10. After a move: the hold, then residency cleanup (P5)

+On `done`, the target writes a cleanup task into the move log: "delete the

+source's copy of `<host>` in `<from>` after `<done + hold>`." The hold is 7

+days by default (`CARLOS_MOVE_HOLD`, §23 Q4).

+**During the hold** the source keeps:

+- the source generation (`from_db`'s location, or `instances/<host>` for

+ a record that was legacy), and the source-keyed receipts and files

+ manifest beside it (S4);

+- the instance files directory and `<host>.env` (box disk);

+- its quarantine file, if any (S19);

+- its config ack object (S9);

+- for an app move, the old store, write-fenced (§11).

+What the hold buys, stated without overselling it. After `done`, the target

+has taken user writes. A **move back** (`carlos move back`, §16) is the

+supported way to return, and it carries the target's data. The kept source

+copy is last-resort evidence for an operator if the target region loses

+data. It is never restored automatically (R4; the 09-24 record's M2).

+**After the hold** the source box deletes all of it and logs the count of

+objects and files removed. The 09-24 record's §6.7 fire-time re-check

+applies, and #199 makes it much simpler. A v2 deletion is keyed to one

+storage key, `<Root>/<host>/<incarnation>/<from_gen>/`, so it runs only if,

+at that moment, the record's `DB` pointer does not name that generation

+(and, for a legacy source, the record has a `DB` pointer at all). A

+boomerang can no longer make the deleted prefix live again, because a

+return lands in a new generation. The files directory is deleted only if

+its `.files-gen` names the source generation's era (it was not replaced by

+a later landing on this box) and this box owns no route for the host. Any

+failure cancels the deletion, with a log line.

+**#199's generation retention.** #199 keeps the newest five generations and

+deletes older ones after a push commits (§5.4 there). After a move, the

+earlier generations live in the source region's bucket, which the target

+box cannot list or delete. So:

+- the retention GC on a box considers only generations in its own bucket;

+- generations a move left in another region are P5's to delete, on P5's

+ schedule, never the GC's;

+- `carlos db rollback --to <gen>` refuses a generation that lives in

+ another region (#199 R3: push and rollback are in-region only), and

+ `carlos db backups` lists such a generation as "in <region>, kept until

+ <date>, not restorable here".

+**#171 guard.** A box role can forge records (§14), so a forged record must

+not be able to schedule a deletion. A cleanup task is honoured only with a

+**console-signed warrant**: the console signs `(move id, host, from, not

+before)` with its release key, which boxes already verify with

+`CARLOS_RELEASE_PUBKEY` and do not hold (`cmd/console/main.go:236-250`).

+The console signs only for a move in its own ledger whose log shows the

+target's commit.

+`carlos move status` and the console show "Ireland copy kept until 9

+October 2026" during the hold and "Ireland copy deleted on …" after it.

+## 11. Stores follow their owner (M3, P8)

+The rule that composes the two kinds of move:

+- **An app-level store follows the app.** It moves in an app move, after

+ the app's instances (§12). An instance move leaves it where it is, and

+ the moved instance reads it across regions (T4 proves it can).

+- **A private store (#412) follows its instance.** It moves inside that

+ instance's move.

+That is M3, and it is consistent with "moving one instance does not change

+the app's region".

+**Prerequisite: a store backend per placement.** `CARLOS_STORE_REGION` is

+read once, console-wide (`cmd/console/main.go:550-560`). Each armed box

+publishes its store region and bucket prefix in its edges file, and the

+provisioner creates a twin in the target's store region. This is the 09-24

+record's §7.1, unchanged.

+**The copy, per store:**

+1. **Provision a twin** in the target store region: same entitlement, new

+ bucket, new credentials (`internal/objstore/provisioner.go`).

+2. **Pre-copy** in delta rounds while everything serves, with copy receipts

+ as the replica mirror uses. The copier is the **move-scoped IAM user**

+ ruled in Q6: Get and List on the one source bucket, Put, Delete and

+ List on the one twin, an `aws:CurrentTime` condition that ends its

+ authority 24 hours after minting, and a reaper for abandoned users. The

+ console mints it only for a move in its own ledger (§14).

+3. **Write fence (the 09-24 record's M1).** A bucket-policy `Deny` on

+ `s3:PutObject` and `s3:DeleteObject` for every principal except the

+ copier. A policy is evaluated per request, so it also stops presigned

+ uploads already handed out. It is the only fence that covers every

+ credential holder.

+4. **Final delta**, then **rebind**: `appconfig.ApplyEnvironment` rewrites

+ the bundle's store coordinates to the twin (default bundle for an app

+ store, the instance's environment for a private store). Config sync

+ bounces running instances, and hibernating ones read it on wake.

+5. **Lift nothing on the old bucket.** It stays write-fenced, read-only,

+ until the hold ends, then P5 deletes it.

+**Where the downtime falls.** For a private store, steps 3 and 4 run inside

+the instance's own cut, between §7.4 steps 6 and 7. The instance is already

+down, so the fence costs nothing extra. For an app store, every instance of

+the app writes it, so the final delta needs all of them held. The app move

+holds the whole app at 503 for the store cut (§23 Q6), measured and shown

+to the owner before they confirm.

+**Rollback.** Before the rebind, a failed store step lifts the fence and

+leaves the binding alone. After the rebind, the store moves forward only.

+## 12. The app move

+### 12.1 Shape

+`carlos move --app ips --to uk`, or the app's Region section in the

+console. The console runs it as a **sequencer**: a console loop over a new

+`app_moves` table in the console DB. The console owns that DB, so the

+sequencer's state survives restarts, and every step is idempotent.

+1. **Validate.** The target is valid, armed, hosting and staging-capable

+ (`apiinstancemove.go:227-235`); eligible for the app (nearest-region

+ §6.1); allowed by the manifest's `regions` (`apiinstances.go:1797`); and

+ not the app's current region. The app is not fleet-placed (the

+ placement column governs its instances' regions, nearest-region §12)

+ and not pooled. The nearest-region switch is effectively on

+ (`CONSOLE_APP_REGIONS=on` and a named home, nearest-region §5.6),

+ because otherwise the app region cannot hold a non-home value.

+2. **Plan.** List the app's records. The instances that move are those

+ whose `region` equals the app's current region and that pass every

+ per-instance rail (§13). Everything else is listed with why it stays.

+ The owner sees the plan before confirming.

+3. **Change the app region** (`apps.region = to`, `region_source = moved`,

+ a new label beside nearest-region §5.1's values). From this moment, new

+ instances created without a region start in the target (the fill,

+ nearest-region §7). That is what "the app region changes, then every

+ instance moves" means in order.

+4. **Move instances one at a time**, each a full instance move with

+ `cut: auto`. One at a time bounds the target's load and keeps every cut

+ window separate.

+5. **Move the app store** (§11), if it has one.

+6. **Done.** The app move's log is the list of its instance moves.

+### 12.2 Partial failure

+- **An instance rolls back.** That instance is back in the old region and

+ serving. The sequencer **pauses** the app move (`paused`, with the

+ reason). The app region stays at the target, so new instances keep going

+ there. The owner chooses: `carlos move resume` (retry the failed one),

+ `carlos move skip --host <h>` (leave it in the old region and continue),

+ or `carlos move --app ips --to <old>` (move the app back). §23 Q5.

+- **An instance goes `stuck`.** The app move pauses and pages, as for

+ `stuck` in §7.6.

+- **The store cut fails before the rebind.** The fence lifts, the store

+ stays where it was, and the app move pauses at "store not moved".

+- **Abort** (`carlos move abort --app ips`) aborts the in-flight instance

+ move if it is still abortable, and stops the sequencer. Instances already

+ moved stay moved. The app region stays at the target. Abort is not undo.

+- **The console restarts mid-move.** The sequencer re-reads `app_moves` and

+ the in-flight instance's record, and continues.

+### 12.3 Composition with the nearest-region spec

+- That spec's R5 says changing an existing app's region is a follow-up,

+ "an owner-only verb that affects future creates only". The app move is

+ that verb. An app with nothing to move (no instances, or all of them

+ elsewhere) is a region change that only affects future creates.

+- An app move writes a non-home `apps.region`, so nearest-region §5.7's

+ rollback floor applies to it unchanged.

+- **An instance move never touches `apps.region`** (nearest-region §12,

+ "Moves").

+- Explicit per-instance placements survive an app move: an instance whose

+ region differs from the app's is not moved, and the plan says so.

+## 13. What does not move, and why

+Each refusal is checked at the door (fast answer) and again by the box

+(records have other writers).

+| shape | answer | why |

+|---|---|---|

+| pooled record (`Pool != ""`) | refused (`apiinstancemove.go:65-68`) | its data lives on the pool's box, not a region box |

+| fleet-placed record or app | refused (`:61-64`; nearest-region §12) | fleets keep state-does-not-follow on purpose |

+| non-hibernating | refused (`:69-72`) | runs outside the activator; nothing can quiesce it |

+| addr-backed, unit-backed, static, status kinds | refused (`:53-60`) | no platform-owned process or data. Static routes are served from the deployment bucket by every edge, so an app move leaves them and has nothing to carry |

+| replica record | refused (`:187-190`, `provision.go:978-980`) | the replica role is placed, not moved (G14) |

+| **home with replicas** | **refused (new)** | Q2. Neither the door nor `MoveRequest` checks `Replicas` today, only `Role`. Phase 1 adds the rail |

+| native sidecar (kind `sidecar`) | refused (`:53-56`) | sidecars carry their own replicated DB (sidecar-members design §4) and need their own move. An app move with sidecars in the source region is refused by default (§23 Q7) |

+| legacy `<host>-sidecar.env` | **refused (new)** | hand-staged on one box; a move would silently drop the sidecar |

+| target not in the manifest's `regions` | **refused (new)** | the app said where it may run. This is also how an operator pins an app that depends on out-of-band host-role grants (S24): set `regions` to its one region |

+| target fails a readiness check (T1 to T8) | cut refused with the check's sentence | §7.3 |

+| instance files over the cap, or containing a special file | refused with the size or the name | §8.1 |

+| another move in flight for the host, or an app move for its app | refused (`provision.go:968-970`) | one at a time |

+## 14. Who may move, and the #171 residue

+- **Owners** of the account (`RoleOwner`), from a session or a person's CLI

+ token. The doors return the family's byte-identical 404 to anyone else,

+ as `handleAppMoveAccount` does for its owner check

+ (`internal/console/appsettings.go:1258-1261`). Members are not owners.

+ Today's move doors admit any member, so this narrows them.

+- **Instance-role service credentials** keep request, cut, abort and status

+ on instance moves (`internal/console/authcli.go:625-628`), unchanged:

+ Tito's router uses them. They cannot start an app move.

+- **Operators** (`Store.IsOperator`, `internal/console/store.go:1858`) can do

+ any of it for any account through an operator route family,

+ `/api/cli/ops/accounts/{sqid}/apps/{app}/…`. Non-operators get 404 there.

+ Every operator act is written to the target account's audit trail.

+**The #171 residue.** Any region box's role can write any record (#171,

+open). A forged `move.to` can already move an instance today, whatever the

+door allows, so M1 does not widen that. What a forged record must not be

+able to do is drive the two privileged acts this spec adds. So the console

+keeps a **move ledger** (console DB: move id, account, app, host, from,

+to, requested by, app move id, outcome) and:

+- mints a store-copy credential (§11) only for a ledger move;

+- signs a P5 deletion warrant (§10) only for a ledger move whose log shows

+ the target's commit.

+A record whose `move.id` is not in the ledger is shown in the console as

+"not started from this console", and nothing privileged follows from it.

+## 15. API

+All additive. Old clients see today's answers.

+| door | change |

+|---|---|

+| `POST …/instances/{host}/move` | body gains `"cut": "auto"`; response gains `move_id`, `from`, and `not_moved` (for example `["app store stays in Ireland with the app"]`). Owners only for people (§14) |

+| `POST …/instances/{host}/move/cut` | refused (409) unless the newest target report passes; the body names the failing check |

+| `DELETE …/instances/{host}/move` | legal until `cut_at`, then 409 "the cut has started; it completes or rolls back on its own" |

+| `GET …/instances/{host}/move` | unchanged by default (titogo depends on it, 09-24 §6.2). `?history=1` adds the newest terminal outcome for 30 days, the target report, timings and `reason` |

+| `POST …/apps/{app}/move` (new) | `{"region":"uk","dry_run":true}` returns the plan; without `dry_run` it starts the app move. People only, owners only |

+| `GET …/apps/{app}/move` (new) | sequencer state, plan, per-instance outcomes |

+| `POST …/apps/{app}/move/resume`, `…/skip`, `DELETE …/apps/{app}/move` (new) | §12.2 |

+| `POST …/instances/{host}/move/resolve` (new) | operator only; §7.6 `stuck` |

+| `/api/cli/ops/accounts/{sqid}/…` (new) | operator mirrors of the above |

+## 16. The CLI (G8)

+One top-level verb, `carlos move`. It is free in `main.go`'s dispatch, and

+`docs/site/cli.md` gains its heading in the same change (`internal/docsite`

+fails the gate otherwise).

+```

+carlos move --app ips --to uk # app move: shows the plan, asks, then runs

+carlos move --host kpmg.bdn.oncarlos.com --to uk # instance move

+carlos move back --host kpmg.bdn.oncarlos.com # instance move to the region it came from

+carlos move status [--app ips | --host <h>]

+carlos move abort [--app ips | --host <h>]

+carlos move resume --app ips

+carlos move skip --app ips --host <h>

+carlos move resolve --host <h> # operators; a stuck move

+```

+- Region names are codes or the home's name (`homeRegionIn`,

+ `apiinstancemove.go:136-141`). Output uses place names (nearest-region

+ §14).

+- `--wait` is the default for a terminal: it prints each transition with

+ its time and exits 0 on `done`, 3 on `rolled_back`, 4 on `stuck`, 5 on

+ refusal. `--no-wait` returns after the request.

+- `--manual-cut` stops at `ready`. `carlos move cut --host <h>` cuts.

+ Without it, the CLI asks for `cut: auto`.

+- `--yes` skips the app move's confirmation, for scripts.

+- Every fact it prints comes from the console (the 2026-08-08 rule). No

+ step needs box access.

+## 17. The console

+**Instance page** (`internal/console/templates/app_instance.html`) gains a

+**Region** section above Danger: the region's place name, "Move to another

+region" with a region select (eligible targets only, each with a reason

+when greyed out), and a confirmation page (M-C10 to M-C13) that says what

+moves, what stays, and the expected 503 window.

+**App settings** (`templates/app_settings.html`) gains a **Region** section

+beside the existing Account section. Its button says "Change region", never

+"Move", because the Account section's "Move to another account"

+(`server.go:1209`) already uses that word.

+**During a move**, both pages show a progress list (M-C20 to M-C27):

+Copying data (with bytes copied), Checking UK (each check, green or the

+failing sentence), Stopping in Ireland, Starting in UK, Done. Each step has

+its time. The Instances table (`app_instances.html:86-91`) shows

+"Moving to UK" in the Region column. The page polls `GET …/move?history=1`.

+**After a move:** "Moved from Ireland to UK on 2 October 2026. 48 seconds

+offline." and the hold line from §10. After a rollback: the reason and

+"Back in Ireland and serving. Nothing was lost." After `stuck`: an

+incident banner for the owner, and the `resolve` button for an operator.

+## 18. Observability (P4)

+- **Move log.** One object per event under

+ `moves/<acct>/<app>/<host>/<move-id>/` in the deployment bucket, named

+ `<unix-nanos>-<actor>.json`. One object per event means three writers

+ (console, source, target) never contend. The target report (§7.3) lives

+ beside them. Events: requested, round (sampled one in ten, plus the

+ last), ready, cut-refused, cut stages, flipped, each landing step,

+ committed, opened, rollback (with the failing step), rolled_back, stuck,

+ cleanup scheduled, cleanup done.

+- **Timings** in the terminal event: prepare, cut (from `cut_at` to the

+ flip), land (flip to open), and **offline**, from `cut_at` to open (or to

+ the source's `ready` after a rollback). Offline is the number owners

+ care about.

+- **Paging:** `rolled_back` is a warning; `stuck` and `landing (overdue)`

+ are incidents; `preparing` with no promotable round for 60 minutes is a

+ warning (finding 1's shape).

+- **Box logs** keep the `moves: <host>: …` shape (`movespass.go`) and add

+ one line per step with its duration.

+- **Member log stream:** each transition is an event on the app's platform

+ stream (`emitEvent`, `internal/activator/logsink.go:31-36`), so `carlos

+ logs` shows the move beside the app's own output.

+## 19. Failure modes

+| failure | detected by | outcome |

+|---|---|---|

+| target lacks the secrets key | T2 | cut refused; source never stops |

+| bundle cannot be opened on target | T3 | cut refused |

+| store unreachable from target | T4, L4 | cut refused, or rollback |

+| target disk too full | T6 | cut refused |

+| target has nothing promoted for its architecture | T7 | cut refused |

+| old target build | T1 (capability in edges file) | door refuses the request |

+| files archive damaged or missing at landing | L1 | rollback |

+| app starts, crashes after binding | L5 settle window | rollback |

+| app needs more than 30 s to start | L5 | rollback. The bar is the same one every wake meets, so the app would fail ordinary wakes too |

+| target box dies during landing | source, at `landing_deadline` | rollback |

+| source dies after the flip, landing succeeds | none needed | done; P5 waits for the box, or for an operator if the box is gone |

+| source dies before rollback finishes | sweep at deadline + 5 min | `landing (overdue)` incident; `resolve` once the box is back |

+| both fail | §7.6 | `stuck`; incident; data intact in the source prefix |

+| staging delete denied on target | cleanup | v2: unreferenced garbage, logged; legacy pre-epoch: logged as a hazard (§7.9) |

+| a stale generation or files directory from an earlier stay on the target | landing | never read: the pointer names the move generation, and L2 moves the old directory aside |

+| a db push requested during a move, or a move during a push | door and box | refused both ways (§26, I-7) |

+| a forged record requests a move | ledger mismatch | no store credential, no deletion warrant; the console flags it |

+| app writes outside its files dir | not detectable | documented loss (§8.4) |

+| app does boot-time side effects | not preventable | they may run twice on a rollback; `CARLOS_MOVE_PROBE=1` lets the app skip them |

+## 20. Tests

+### 20.1 Unit and integration

+- `MoveState` transitions, including the new states, every illegal edge

+ (409 matrix), and old-record parsing.

+- Archive: deterministic bytes for one input; special files refused;

+ truncation and chunk-reorder detected; wrong key refused; per-file hashes

+ checked on extraction; flat-to-nested and nested-to-flat extraction.

+- Target report: each of T1 to T8, failing and passing, against fakes and

+ against MinIO for T4.

+- Landing: L1 to L7 each failing in turn leads to rollback, never to open.

+ An L6 lost to a rollback never opens.

+- Reconcile mints `landing` for a landing record, and `ready` otherwise.

+- The source's `rolling_back` arm with the `moving` row present and with it

+ retired.

+- Timers carried: a once-timer due during the cut fires once after open; a

+ declared schedule keeps its `next_at`.

+- Door: owner-only, the operator family, home with replicas, `regions`

+ allow-list, legacy sidecar env.

+### 20.2 The state-machine simulation

+A Go test that models the console, the source box, the target box, the

+sequencer, the record store (with CAS), the source and target buckets, a

+client issuing writes through whichever edge serves, and a clock. Each step

+of each actor is an atomic action. A seeded scheduler picks the next

+action, and can crash any actor (losing its memory, keeping its disk and

+buckets) or delay it. Faults are injected at the same points as §19: each

+landing check failing, box death, deadline expiry, denied deletes.

+After every action it asserts INV-1, INV-2, INV-4, INV-4b and INV-6. At

+the end of every run it asserts INV-3 (every acknowledged client write is

+in the authoritative copy), INV-5 (every start saw the authoritative files

+version), INV-7 (terminal within the bound) and INV-8.

+The model carries #199's storage: generations keyed by incarnation, the

+record's `DB` pointer, `.lineage` and the two directory markers, and the

+epoch, which the scheduler can write at any moment, so legacy, first-v2

+and v2 moves all occur. Its runs include moves interleaved with a

+`carlos db push` request, and boomerangs (ie→uk→ie, including with an

+earlier move's generation and files directory still in place).

+- 2,000 seeds in the ordinary gate; 200,000 in a nightly job.

+- **Mutation checks**, run in a copy of the tree: open before commit; skip

+ the cut's report re-check; flip `region` and `DB` in two writes; let the

+ target restore from a generation the pointer does not name; let rollback

+ leave `DB` at the move generation; let a landing reuse an existing files

+ directory; extract files after the probe; let the source purge the files

+ directory at the cut. Each mutation must

+ make the simulation fail within the gate's seed count. A mutation it

+ cannot catch is a defect in the simulation.

+- The simulation drives the real transition functions

+ (`provision.Move*`) and the real worker step functions through injected

+ interfaces, not a re-implementation of them. A fake that agrees with

+ itself proves nothing.

+### 20.3 Cross-version

+An old target (no capability) is refused at the door. A v0.22 console

+refuses abort on every new state (P0's allowlist). A new console against

+old boxes refuses every move.

+### 20.4 Rehearsal with an ips-shaped app

+The `bdf/hello` rehearsal passed because hello has none of the state that

+failed. The rehearsal app, `moveprobe` (a test app in this repo), has every

+piece of it:

+- at first boot it writes `instance.key` into its files directory and seals

+ a row with it; every boot verifies the seal and **refuses to start** on a

+ mismatch, so a missing or regenerated key fails the health check;

+- a store binding it requires at boot (refuses to start without it), and an

+ object it writes and reads back on every boot;

+- a sealed config secret it requires;

+- a declared schedule and a once-timer, each logging its runs to the DB;

+- a config switch, `MOVEPROBE_FAIL_IN=<placement>`, that makes it fail

+ health in one region, to force a rollback.

+Rehearsals, each with timings recorded in the move log:

+1. dev deployment: moveprobe IE to uk, done; then forced failure, rolled

+ back; then uk to home after Phase 3.

+2. flagship, with scoped approval: a disposable moveprobe instance under the

+ operator's account, the same three runs.

+3. flagship, with scoped approval: a disposable ips instance in `bdn`,

+ seeded through the app's own seed, with a key and a store. This is the

+ shape that failed. (§23 Q9.)

+4. Only then kpmg, at a time Paul chooses.

+Each run passes only if the key verifies, the object reads back, the

+secret is present, every timer ran exactly once around the cut, and the

+offline time is recorded.

+## 21. Rollout and gates

+0. **#199's release N first** (its §7): every box and console understands

+ `DB`, `StorageKey`, `.lineage` and the epoch. Phase 1's code reads and

+ writes storage only through `StorageKey` (§26, I-2), so it is built on

+ top of N, not beside it. Its own rollback floor (#199 §7 step 4) then

+ covers moved records too: a binary older than N would drop a moved

+ record's `DB` pointer and send it back to its first generation.

+1. **Consoles first.** They learn the new states, the cut door's report

+ check, owner-only doors, and the ledger. Moves keep working as today

+ against old boxes, because the door refuses any target that does not

+ publish `moves_schema >= 2`. In practice that refuses every move until

+ step 2.

+2. **Boxes, every region.** The source and the target roles ship together.

+ A box publishes `moves_schema: 2` only when it runs the code. Mixed

+ fleets are safe: the door checks both boxes.

+3. **Infra check, per deployment:** every region box publishes a

+ `secrets_key_fp` equal to the console's. Where not, stage the key (the

+ SSM note in §6). This is production configuration and needs scoped

+ approval.

+4. **Rehearsals** (§20.4) in order. Each one gates the next.

+5. **Owner self-serve** turns on with `CONSOLE_MOVES_SELF_SERVE=on`, and

+ only after Phase 3 (P5 deletion) is live, so a move that promises

+ residency delivers it. Until then the doors serve operators and

+ instance-role services only.

+6. **App moves** turn on after Phase 4 and its rehearsal: an app with two

+ moveprobe instances and an app store.

+## 22. Phases

+Each phase lands dormant and is useful alone.

+|---|---|---|---|

+| **1. Safe instance moves** | journal check of §1.2b's cause; files archive (§8) with `<host>.env` and timers, keyed by incarnation and generation; target readiness T1 to T8 and the cut's re-check (§7.3); `landing` with the durable fence, L1 to L7, automatic rollback and `stuck` (§7.5, §7.6); `MoveState` fields and the generation-carrying flip and rollback (§7.1, with #199 §4); the legacy path (§7.9) if Q11 keeps it; edges file `moves_schema` and `secrets_key_fp`; new rails (home with replicas, legacy sidecar env, `regions`, no move during a push); the minimal move log (events and `reason`); the simulation (§20.2) | #199 release N built (rolled before Phase 1 rolls); the §26 interface points agreed by both specs | rehearsals 1 to 3 pass; kpmg moves with Paul's approval |

+| **2. Owner surface** | `carlos move` (G8); console Region sections and progress; auto-cut; owner-only and operator doors; the ledger; P2 holding route (503 for custom domains and the source's holding answer); full P4 log, history, paging; copy review | Phase 1 | an owner can move an instance without help, with `self-serve` still off |

+| **3. Home and residency** | P3 (§9); `carlos move back`; P5 hold and deletion with signed warrants (§10) | Phase 2; infra approval for the home staging grant | rehearsal 1's uk-to-home run passes; a hold expires and deletes; `self-serve` on |

+| **4. App moves and stores** | the sequencer (§12); `apps.region` writes; store backend per placement; private and app store moves with the write fence and the move-scoped credential (§11) | Phase 3; nearest-region spec built | the two-instance app rehearsal passes |

+Phase 1 is the smallest set that makes an ips-shaped move safe. Without

+the archive the key is lost (finding 2a). Without T2 and L3 the config is

+missing (finding 2b). Without L5 and §7.6 a bad target is an outage rather

+than a 503 and a rollback (M2). Without the simulation the next

+interleaving is found in production. Everything else is surface,

+convenience or residency, and none of it is needed for a move to be safe.

+## 23. Open questions for Paul

+Each is a choice, with a recommendation.

+1. **What the archive carries.** (a) Only the fixed set in §5 (the files

+ directory, `<host>.env`, timers). (b) Also paths an app declares in its

+ manifest. **Recommend (a).** One rule is easy to document and to

+ verify, and every app we run fits it. (b) can be added later without

+ changing the format.

+2. **Archive size cap.** (a) 256 MiB, refused above it. (b) 1 GiB. (c) No

+ cap. **Recommend (a).** Key files and tokens are kilobytes. Anything

+ large belongs in object storage, and the cap keeps the cut window

+ predictable.

+3. **Cut timing for person-started moves.** (a) Auto-cut when ready and the

+ target passes, the default in the CLI and the console. (b) Always ask.

+ **Recommend (a)**, with `--manual-cut`. The API default stays manual,

+ because titogo holds its own freeze before cutting.

+4. **Hold period.** (a) 7 days. (b) 30 days. Separately, may an owner end

+ the hold early ("delete the Ireland copy now")? **Recommend 7 days, and

+ yes to early deletion for owners**, as an explicit, confirmed act.

+5. **An app move when an instance rolls back.** (a) Pause, keep the app

+ region at the target, owner decides. (b) Continue with the rest and

+ report. (c) Revert the app region. **Recommend (a).** It never flaps the

+ app region, and the owner sees the failure before more cuts happen.

+6. **Downtime for an app store's final copy.** (a) Hold every instance of

+ the app at 503 for the store's final delta. (b) Keep instances up with

+ the bucket write-fenced, so uploads fail with storage errors for that

+ window. **Recommend (a).** A 503 is honest and retried; (b) shows users

+ app errors such as "File storage is unavailable". The plan shows the

+ expected length before the owner confirms.

+7. **Apps with native sidecars in the source region.** (a) Refuse the app

+ move until sidecar moves exist (Phase 5). (b) Move the instances and

+ leave the sidecars, listed in the plan. **Recommend (a).** A sidecar

+ that talks to its app across regions is a surprise nobody chose.

+8. **Pre-warming the target** (restore staging into a scratch DB during

+ `ready`, so the cut only applies the last delta). (a) Later, if

+ rehearsals show the window above about a minute. (b) In Phase 1.

+ **Recommend (a).** It adds a second restore path to Phase 1, which is

+ exactly where the risk should not grow.

+9. **Rehearsal subject on the flagship.** (a) moveprobe, then a disposable

+ ips instance in `bdn`, then kpmg. (b) moveprobe, then kpmg. **Recommend

+ (a).** Finding 3 is that a rehearsal must have the moved app's shape,

+ and moveprobe has the shape we know about but not ips's own startup.

+10. **Members.** Today's move doors admit any member of the account. M1

+ names owners. (a) Owners only, as written in §14. (b) Owners plus a new

+ "can move" grant for members. **Recommend (a)**, matching app delete

+ and the account move.

+11. **Moves before #199's epoch exists.** (a) Refuse every move until the

+ epoch is written, so production only ever runs the v2 landing. (b)

+ Support the pre-epoch legacy flow of §7.9 as well. **Recommend (a).**

+ #199's rollout writes the epoch as soon as every box and console runs

+ release N, which Phase 1 needs anyway. (b) keeps "staging wins" and a

+ rollback that has to delete a live-looking lineage in the target, which

+ is the hazard the generations exist to remove. Choose (b) only if kpmg

+ must move before #199's epoch can be written. Either way, a legacy

+ record moved after the epoch takes the first-v2 path of §7.9.

+12. **Where a landed generation lives** (#199 §4's open point, I-1 in

+ §26). (a) The pointer names the staging root (`Root: "moves"`), with no

+ copy. (b) Landing copies the staged generation into

+ `instances/<host>/<incarnation>/<to_gen>/` before the probe, and the

+ flip points there. **Recommend (b)**, for the reason in I-1: under (a)

+ every moved instance's live replica sits in the one prefix that every

+ other region's box role may write and delete.

+## 24. Copy (pending copy review)

+Drafts. None of these is approved. Every string goes through the

+copy-review gate before it is written into code or templates. Place names

+come from nearest-region §14. `<…>` marks a value.

+### CLI

+| id | string |

+|---|---|

+| M-C1 | Moving <host> from <from> to <to>. |

+| M-C2 | Copying data. <host> keeps serving while this runs. |

+| M-C3 | Copied <bytes>. Waiting for the copy to catch up. |

+| M-C4 | Checking that <to> can run <host>. |

+| M-C5 | <to> is ready. Starting the cut. <host> will answer 503 for about <estimate>. |

+| M-C6 | Stopped in <from>. Starting in <to>. |

+| M-C7 | Moved. <host> is served from <to>. It was offline for <seconds> seconds. |

+| M-C8 | The <from> copy is kept until <date>, then deleted. |

+| M-C9 | <to> could not start <host>: <reason>. It is back in <from> and serving. Nothing was lost. |

+| M-C10 | <host> could not start in <to> or in <from>. It is offline. An operator has been paged. |

+| M-C11 | Ready to cut. Run carlos move cut --host <host> when you are ready. |

+| M-C12 | This move has started its cut. It will finish or move back on its own. |

+### App move

+| id | string |

+|---|---|

+| M-C13 | Moving app <app> to <to>. New instances start in <to> from now on. |

+| M-C14 | <n> instances will move, one at a time. Each is offline for about a minute. |

+| M-C15 | <n> stay where they are: |

+| M-C16 | <host>, already in <region>. |

+| M-C17 | Its files will move to <to> after the instances. The whole app is offline for about <estimate> while they finish copying. |

+| M-C18 | Paused. <host> could not move: <reason>. It is serving in <from>. Run carlos move resume --app <app> to try again, or carlos move skip --app <app> --host <host> to leave it there. |

+| M-C19 | Moved app <app> to <to>. |

+### Refusals

+| id | string |

+|---|---|

+| M-C30 | <to> cannot take this move yet: <check>. |

+| M-C31 | <to> cannot read this app's configuration. An operator needs to install the deployment's secrets key there. |

+| M-C32 | <to> cannot reach this app's storage. |

+| M-C33 | <to> does not have enough free disk for <host>. |

+| M-C34 | This instance has replicas. Instances with replicas cannot move yet. |

+| M-C35 | This app only runs in <regions>. Change that in its instances settings first. |

+| M-C36 | <host> keeps a file this move cannot copy: <path>. Only regular files and folders move. |

+| M-C37 | <host> keeps <size> of files beside its database. Moves carry up to 256 MiB. Keep large files in object storage. |

+| M-C38 | <host> is already moving. Wait for it to finish, or run carlos move abort --host <host>. |

+| M-C39 | Moves into <home> are not available yet. |

+| M-C40 | This app has sidecars in <from>. Apps with sidecars cannot move yet. |

+### Console

+| id | string |

+|---|---|

+| M-C50 | Region |

+| M-C51 | Served from <region>. |

+| M-C52 | Move to another region |

+| M-C53 | Change region |

+| M-C54 | Moving copies this instance's data to <to>, stops it here, and starts it there. It is offline for about a minute. If it cannot start in <to>, it comes back here on its own. |

+| M-C55 | What moves: the database, the files beside it, its settings and its scheduled jobs. |

+| M-C56 | What stays: the app's files storage stays in <region> with the app. |

+| M-C57 | Move <host> to <to> |

+| M-C58 | Moving to <to> |

+| M-C59 | Copying data |

+| M-C60 | Checking <to> |

+| M-C61 | Stopping in <from> |

+| M-C62 | Starting in <to> |

+| M-C63 | Done |

+| M-C64 | Moved from <from> to <to> on <date>. Offline for <seconds> seconds. |

+| M-C65 | The <from> copy is kept until <date>. |

+| M-C66 | The <from> copy was deleted on <date>. |

+| M-C67 | Back in <from> and serving. Nothing was lost. <to> could not start it: <reason>. |

+| M-C68 | This move needs an operator. The instance is offline. |

+| M-C69 | Not started from this console. |

+## 25. Gaps this spec closes, against today's code

+| # | gap | where |

+|---|---|---|

+| X1 | Instance files directory not carried, not backed up | `movespass.go:476-494`; no reference anywhere in the repo |

+| X2 | `<host>.env` not carried: new admin token, box-local overrides lost | `prepare.go:49`; `admintoken.go:115` |

+| X3 | Config rendered only where `CARLOS_SECRETS_KEY` is staged, and absence is silent at spawn | `agent.go:881-886`, `:1812`; `backend_exec.go:1146-1149` |

+| X4 | Nothing checks the target before the source stops | `movespass.go:415-505` |

+| X5 | Target opens at mint, with no health gate | `reconcile.go:374`; `edge.go:595` |

+| X6 | No automatic rollback; recovery was a hand edit of the record | #175, 2026-10-02 post |

+| X7 | Home with replicas passes the move gate (Q2 unimplemented) | `apiinstancemove.go:187-190`; `provision.go:966-983` |

+| X8 | The manifest's `regions` allow-list is not applied to moves | `apiinstancemove.go:179-237` reads no manifest |

+| X9 | Timers lost or reset (G15) | `registry.go:565-579`; `converge.go:223-241` |

+| X10 | Any member can move | `authcli.go:625-628` |

+| X11 | The isolated layout pass, as designed, strands `<DataDir>/<host>/` | isolated-exec design §2 item 1 |

+| X12 | Moves to home refused | `apiinstancemove.go:203-209`; `movespass.go:345`; `backend_exec.go:989-993` |

+| X13 | Source replica, files and env never deleted | `movespass.go:492` (DB artifacts only) |

+## 26. Interface points with #199 (replica ownership)

+The #199 session will not change `movespass.go`, `promoteStagedReplica`

+or `MoveComplete` until the two specs agree. These are every point where

+they meet. Each says who owns it and what this spec needs. Both specs

+should be reviewed against this list together.

+|---|---|---|---|

+| I-1 | **Where a landed generation lives.** #199 §4 says `DB = {Gen, Root: "moves"}` and no copy; the coordinator reports it as still undecided. | #199 | **Recommend a copy into `instances/<host>/<incarnation>/<to_gen>/` on landing**, as a step between L4 and L5, with the flip writing `Root: "instances"`. The reason is IAM. Every sibling region's box role holds `s3:PutObject` and `s3:DeleteObject` on `moves/*` in this bucket (`infra/modules/region-host/main.tf:217-221`), so under `Root: "moves"` a moved instance's **live** replica sits in the one prefix that every other box, and anything that obtains a box role (#171), may overwrite or delete. The copy is an in-bucket server-side copy into a key nothing else uses (the move id is in it), so it reintroduces none of the staging-wins hazards, and it costs what today's promotion costs (`backend_exec.go:1580-1606`). If #199 keeps `Root: "moves"`, this spec works unchanged (L5 restores whatever the pointer names), and the IAM exposure is recorded as accepted. |

+| I-2 | **One function for every location.** `movespass.go:135`, `:364`, `:365` and `:369` build the checkpoint prefix, the mirror source, the staging destination and the receipts key from the host alone. | #199 defines; this spec calls | The moves code computes the source with `StorageKey(rec, epoch)` (#199 §2.3) and the destination with a new `provision.MoveStagingKey(rec, toGen)` that #199 defines beside it, so the layout lives in one package. The cut's checkpoint goes into the source storage key. |

+| I-3 | **The flip, the commit and the rollback CAS.** | shared | `MoveComplete` writes `Region = to`, `DB = {to_gen, Root}`, `state = landing` in one CAS, guarded by move id and `DB == from_db`. A new `MoveCommit` (target) clears `move`, guarded by move id and `DB.Gen == to_gen`. A new `MoveRollback` writes `Region = from`, `DB = from_db`, `state = rolling_back`, with the same guard. #199's push commit is guarded by request id, incarnation and `FromGen` (#199 §5.4 step 5). The two families must never both succeed on one record: see I-7. |

+| I-4 | **Generation allocation for a move.** #199 §4 says a move allocates `g<seq+1>-<moveID>` but not who. | this spec | The console allocates `to_gen` in `MoveRequest` from `from_db`'s sequence (0 for legacy or pointerless), with the console-minted move id as the nonce. The move id's alphabet must be valid in #199's `Gen` grammar; #199 states that grammar. |

+| I-5 | **Incarnation of a legacy record with zero CreatedAt.** | #199 | `provision.Incarnation(rec)` must return the same value on every box for such a record, or #199 must say that a zero-CreatedAt record is refused a move until it is backfilled. The first-v2 landing (§7.9) keys the staged copy, the archive and the files markers by it. |

+| I-6 | **The files directory and its markers.** #199 §2.5b gives the nested `InstanceDir` an `.incarnation` marker and moves a mismatch aside to `.retired/<old-incarnation>/`. | shared | (a) Extend the same check to the flat `<DataDir>/<host>/`, which this spec makes a platform-defined path (§5); a recreated flat host otherwise inherits the deleted instance's `instance.key`. (b) This spec adds `.files-gen`; #199's push must leave it unchanged. (c) `.retired/` names need the generation too, because one incarnation can be moved aside more than once by successive landings on one box: `.retired/<incarnation>-<files-gen>/`. (d) #199's wake check must treat a directory without markers as "not landed yet" during `landing`, not as a mismatch to retire, because L2 writes the markers last. |

+| I-7 | **A push and a move never overlap.** | shared | `MoveRequest` refuses while the record has an in-flight push request; #199's push request refuses while `move != nil`, in any state including `stuck`. The route statuses stay distinct: `db-push` is #199's, `moving` and `landing` are this spec's. #199's restart rule ("a route left at `db-push` with no in-flight request is released") never touches a `landing` row, and the landing worker's repair (§7.5, L7) never touches a `db-push` row. |

+| I-8 | **Receipts and the source's files manifest.** | this spec, on #199's key | `move-receipts/<host>/<incarnation>/<from_gen>.json` (#199 §4) and `….files.json` beside it (§8.2); legacy records keep `move-receipts/<host>.json`. Both are cleared at a terminal state, and P5 deletes anything left with the source generation. |

+| I-9 | **Purge with errors.** | #199 | The cut (`movespass.go:492`) and the target's rollback use `PurgeLocalDBArtifactsErr` (#199 §5.6), which removes `.lineage` first; an error fails the pass and is retried. Neither purge touches the files directory. |

+| I-10 | **Registry columns and retire-on-identity-change.** | #199 | The landing row is minted with #199's `incarnation`, `storage_key` and `storage_gen` and status `landing`. After the flip, the source's row can be retired twice over (out of scope by region, and a storage-key mismatch under #199 §2.6). Both are idempotent, and **#199 §2.6's retire must purge only the database, never the files directory**, which is the rollback's material on the source (§7.4 step 8). On rollback the source row's storage key equals the record's again, so no retire fires. |

+| I-11 | **Generation retention across regions.** | #199 | The keep-five GC (#199 §5.4) considers only its own bucket. Generations a move left in another region are P5's (§10). `carlos db rollback` and `carlos db pull --generation` refuse an out-of-region generation (#199 R3) and `carlos db backups` labels it. |

+| I-12 | **The epoch and moves.** | shared | Q11: refuse moves until `control/storage-v2.json` exists, or support §7.9's pre-epoch flow. Either way #199 §7's rollback floor (never below release N once the epoch exists) also protects moved records. |

+| I-13 | **One record listing per pass.** | shared | The moves pass's source and target roles share #199 §5.7's single `ListAllRecords`. Every destructive step re-reads its own record, as both specs already require. |

+| I-14 | **Reads during a move.** | #199 | The pull door (`resolveReplica`) refuses an instance in `landing` or `rolling_back`, because the pointer names a generation that is not yet authoritative (INV-4). In other states it serves the pointer, as #199 specifies. |

+| I-15 | **Staging wins, retired.** `promoteStagedReplica` (`backend_exec.go:1580`) is called from `Restore` (`:290`). | #199 removes; this spec agrees | It runs only for a legacy row on a pre-epoch deployment (§7.9). For v2 rows `Restore` never lists `moves/<host>/`. |