Eleven → Carloku: customer-fleet migration

Date: 2026-08-29 Status: Approved design (Paul). Revised 2026-08-30 twice: first after an adversarial review (Fable) found three Criticals in the original wave 1, then again after Paul reframed the work into two phases — prove it greenfield, then move production hosts one at a time. That reframe dissolves most of what the review found, so this document supersedes the box-conversion plan entirely.

Refreshed 2026-09-05 against main, after the spec sat unmerged on branch eleven for a week while the platform moved 269 commits. Three passages had gone stale and are amended in place, each marked and dated where it sits: phase 5's tick-cadence argument (void — #371 landed), gap 3 (demoted), and gap 6 (closed — the file it names was already deleted). Nothing else changed; the design and its decisions stand as approved.

Account: eleven (sqid bdr) on https://console.carloku.com Supersedes: infra/eleven-fleet.md, which was factually wrong about Eleven's starting state (see "The reframe"). That runbook was deleted from main in the open-source redaction pass (2adc71f, 2026-08-27), so this document is now the only account of where Eleven starts.

The reframe this spec exists for

infra/eleven-fleet.md says Eleven "moves from Tito's own self-hosted CARLOS install onto Carloku's console" — a control-plane re-point, "not a data migration: there is no data to migrate." The first half is wrong.

Eleven is not on CARLOS at all. Checked 2026-08-29:

• home.elevenmessenger.com resolves to 89.167.122.227 and serves 200 with no X-Carlos-Version header. It is not behind a CARLOS edge.
• The repo carries a complete operator surface of its own: deploy/ops with build/sign/publish, edge|beta|stable channels, fleet sweeps, canaries and a dev mesh; lchat router with its own cert cache and wildcard DNS-01; activator.go with its own hibernation, litestream replication and single-writer lease.
• CARLOS's equivalents were ported from that code — commit 845ffd4, "activator, releases/channels, fleet ops (ported from Eleven + titogo)".

So this is a first-time onboarding onto a control plane that is Eleven's own machinery, generalised. That is why the overlap is total rather than partial, and why most of the work is deleting Eleven's operator surface rather than translating it.

The second half of the old runbook's claim survives, for a different reason than it gave, and it is now verified rather than inherited — see D4.

Verified starting state (2026-08-29)

Every line here was checked, not assumed.

| Fact | How it was checked | |---|---| | Account eleven (bdr) exists on Carloku | carlos auth whoami | | It has zero apps | carlos instances list -account eleven → "account eleven has no apps" | | It has no fleets | carlos fleets ls -account eleven → "no fleets" | | Production is two boxes: one Hetzner, one Fasthosts, neither EC2 | Paul, 2026-08-29 | | Boxes serve directly; no CARLOS edge in front | curl -sI https://home.elevenmessenger.com | | Eleven's AWS account is 427880590669 | sts get-caller-identity on the vault's dev-agent key | | Hibernate bucket eleven-prod-instances-427880590669, prefix prod/ | vault item note + s3api list-objects-v2 | | Layout is prod/<host>/ by FQDN, plus prod/mesh/ | s3api list-objects-v2 --delimiter / | | 31 instances: 23 *.11ch.at, 5 *.luan.chat, 2 *.elevenmessenger.com, 1 chat.paulca.com | same listing | | All four zones are on DNSimple | dig NS on each | | Fleet API surface (#73) merged | PR #145 MERGED 2026-08-11 | | region-remote-host's deployment_bucket_arn is optional | infra/modules/region-remote-host/variables.tf:11 | | carlos apps place / apps create --place exist | apps.go:111, apps.go:382 | | Sidecar members not merged | PR #223 OPEN, last updated 2026-08-18; re-checked 2026-09-05 — still unmerged, no sidecar package on main | | Eleven already runs a tofu-managed AWS node: EC2 t4g.small arm64, EIP, S3 bucket, IAM role | infra/dev-mesh/{compute,storage,route53,dns}.tf | | dev.11ch.at is delegated from DNSimple into a Route53 zone, live at 54.154.95.205 | aws_route53_zone.dev + dnsimple_zone_record.dev_delegation; dig NS | | https://amadan.net/eleven/infrastructure exists and is empty | git ls-remote → exit 0, no refs | | lchat relocate is mesh-internal and cannot target a CARLOS box | relocate.go:1-30 (home_region, fanOutMeshRoute, candidatesFor) | | Status bucket eleven-status-427880590669 is 521 objects / ~270 MB, of which prompts/ is 422 / ~270 MB — the site itself is ~620 KB | aws s3 ls --recursive --summarize | | carlos ledger provides append / blob / publish / verify, credential-free verification | carlos ledger -h |

So every precondition infra/eleven-fleet.md lists as "pending merge" has landed, and the two it assumes are already met — apps created, fleet existing — are both false.

Decisions

| # | Decision | Rationale | |---|---|---| | D1 | Fresh boxes, never converted in place. The two existing production boxes are retired when empty, not migrated. | Converting a live box in place carried the review's worst findings: a shared registry file, a fleet-wide lease drain, and a ports 80/443 handover under production traffic. A fresh box has none of them. | | D2 | Sidecars are proven in phase 1 and hardened in phase 3. | Sidecars do not spin up today (Paul) and the reason is unknown, so a greenfield box is where the answer is cheapest to find — and no production behaviour can regress while looking. | | D3 | Sidecars run on a third fleet box with CARLOS_INBOUND=none, on ordinary outbound-only networking — no NAT gateway, admin access retained. | Eleven's own docs/bots.md: the bot host transiently holds thread plaintext and keys, so "root on the chat server reads every bot thread." Moving it off the serving boxes restores the guarantee. The full private-subnet posture is not needed for that and costs the break-glass channel. | | D4 | Boxes serve directly in phases 1-2. Carloku is control plane only. | A fleet box with no deployment bucket has its cert issuance gate cleared and "issues and keeps its own certs, as before" (internal/edge/certsync.go:117). No Eleven traffic transits Carloku, and no metadata leaves Eleven's boxes. | | D5 | Edge-fronted ingress is phase 4, gated on two written entry conditions. | Fleet routes cannot be steered today, and the WebSocket hop is unmeasured. See "Phase 4". | | D6 | The sidecar box goes in Eleven's AWS account (427880590669). | It is an ordinary hashicorp/aws EC2 instance, it sits beside the buckets that plane already holds, and region-remote-host is close to the right module with deployment_bucket_arn omitted. | | D7 | Two phases. Phase 1 proves the whole product greenfield on a throwaway namespace; phase 2 moves the 31 production hosts. | Phase 1 turns the review's unknowns into rehearsals at zero risk: it proves release delivery, teaches the manifest shape at four hosts instead of 31, and answers why sidecars do not spin up today. | | D8 | Phase 2 moves hosts one at a time by DNS, keeping each address identical. | docs/multi-edge.md: the address is the identity, and changing it re-keys every member. A per-host A record overrides the wildcard, so hosts cross individually and reversibly. lchat relocate cannot do this — it is mesh-internal (home_region, fanOutMeshRoute, candidatesFor) and a CARLOS box is not a mesh peer. | | D9 | All new boxes are AWS EC2 in Eleven's account, tofu-managed from a root at https://amadan.net/eleven/infrastructure. | Tito covers the bill for now (Paul, 2026-08-30). The tofu root is Eleven-owned and lives outside the platform repos, per the customer-fleet rule. The repo exists and is empty. |

Topology

DNS stays pointed at Eleven's boxes. Each box runs carlos agent — edge plus activator — issues its own wildcard certificates, and serves directly. Carloku is reached outbound over the tunnel and carries control only: releases, channels, config, placement, fleet visibility, wake.

Certificates. A fleet box carrying no deployment bucket of its own has armCertIssuance clear the issuance gate, because there is nothing to publish to and nothing to mirror from; it issues for itself. Wildcards come from newWildcardFromEnv under the same gate.

DNS provider — a constraint, not a coincidence. internal/edge/dns.go selects exactly one provider per box from CARLOS_DNS_PROVIDER. Eleven serves four zones (11ch.at, elevenmessenger.com, luan.chat, paulca.com) and all four are on DNSimple, so CARLOS_DNS_PROVIDER=dnsimple plus CARLOS_DNSIMPLE_TOKEN covers the whole estate. Had any zone been elsewhere, a directly-serving box could not have issued both wildcards and D4 would not have been available. Record this: adding a zone at a second DNS provider breaks the direct-serve path.

The token is the vault's lchat DNSimple DNS-01 token (11ch.at — Eleven Messenger acct 176940) — password slot, per deploy/vault-items.sh.

Three wildcards, not four — deliberately. CARLOS_WILDCARD_DOMAINS lists 11ch.at, elevenmessenger.com and luan.chat. It omits paulca.com, because Eleven serves exactly one host there (chat.paulca.com) and a wildcard would grant the box authority to mint certificates for the whole of a personal domain it has no other business in. An exactly-named route gets an exact certificate through the ordinary registry host policy, which is the correct outcome. Any future member custom domain follows the same rule: one host means an exact name, a zone of instances means a wildcard.

Phase 1 — the greenfield proving ground

A CARLOS-native Eleven, standing beside production and touching none of it. Everything here is throwaway: a namespace nobody's keys are bound to, on a box that can be destroyed and rebuilt.

Amended 2026-09-05 — phase 1 starts on the shared fleet, not a fresh box

Paul, 2026-09-05: "Since Eleven is decentralised anyway, I think the first goal will be getting a running instance on https://console.carloku.com/a/eleven/ and then we go from here migrating the other apps."

This is a smaller and better first step than the box below, and it reorders phase 1 rather than replacing it. The original phase 1 opens with infrastructure — port infra/dev-mesh into a new tofu root, stand up an EC2 box, point it at carlos agent — and only then proves the product. But none of phase 1's five proofs actually need Eleven to own the box. Account eleven (bdr) has no fleet, so an app created in it lands on the shared Carloku fleet, which already exists and is already running. That defers the tofu root, the AWS spend and the eleven/infrastructure repo out of the critical path entirely.

It also matches the product: Eleven is decentralised, one instance per space, so a single instance is a working Eleven — not a stub standing in for one. There is no all-at-once cutover to rehearse.

Two findings from 2026-09-05 that make this cheap:

1. The launch contract already matches, exactly. CARLOS spawns an app as exec.Command(bin, "--socket", rt.Socket, "--db", dbPath) (internal/activator/backend_exec.go:691). Eleven spawns its own instances as exec.Command(bin, "--socket", rt.Socket, "--db", dbPath) (messenger/activator.go:1218) — the same line, because CARLOS's activator was ported from Eleven's. lchat needs no code change to be launched by a CARLOS box.

1. Verified by running it. lchat --socket <path> --db <path> on an empty directory boots, creates its SQLite database, prints an admin link, logs lchat listening on unix:<path>, and answers HTTP 200 with the app shell over that socket. From nothing, with no config and no environment.

The one mismatch found: architecture. carlos ship --target defaults to linux-arm64 and the shared fleet is Graviton, but the binary committed to Eleven's repo is lchat-linux-amd64. Phase 1 needs an arm64 build. This is a build-matrix line, not a design problem.

So phase 1 now runs in two steps. Step A — one instance on the shared fleet, which proves release delivery (proof 1), the manifest shape (proof 2) and that the product works on CARLOS (proof 3), with no infrastructure at all. Step B — the box below, which is still needed for the proofs step A cannot reach: sidecars (proof 4) and the disposition of lchat router's hooks and links (proof 5), both of which want a box Eleven controls. Everything from "The infrastructure already exists in the right shape" onward is step B.

Step A is done — 2026-09-05

Eleven runs on Carloku. proving.bdr.oncarlos.com serves Eleven's app shell over HTTPS from the shared fleet, on a binary shipped through carlos.

The whole path, for the record:

ARCH=arm64 ./deploy/build-linux.sh # in Eleven's repo → lchat-linux-arm64 carlos apps create --account eleven --app proving carlos deploy --account eleven --app proving --target linux-arm64 \ --version ffd3de0 lchat-linux-arm64 carlos instances enable --account eleven --app proving carlos instances create --account eleven --app proving \ --host proving.bdr.oncarlos.com --channel edge

What that proves.

• Proof 1, release delivery, end to end. deploy shipped ffd3de0 and promoted it to edge; the box's reconcile pass materialised the route and the adopt installed the binary. The edge stamps x-carlos-version: ffd3de0.
• Proof 2, the manifest shape. instances enable pinned the app to bdr.oncarlos.com and the one-label rule held: proving is one label under the account's platform subdomain, and the console checked the host against the manifest before minting the record.
• Proof 3, the product runs on CARLOS. The box logs lchat listening on unix:/run/carlos/proving.bdr.oncarlos.com.sock (db: /data/proving.bdr.oncarlos.com.db), and the served page carries Eleven's own CSP — connect-src naming home.elevenmessenger.com, the giphy and youtube-nocookie sources — not a platform placeholder. Signing in and creating a space is the remaining half of this proof and needs a person.
• Hibernation works, unprompted. The record read Setting up, then served, then went to Asleep on its own. The instance woke for a request and slept again with nothing configured for it.

Two findings worth carrying.

1. The admin link names the wrong host. On first boot lchat prints http://localhost:1747/admin#token=… because it has no idea what hostname fronts it — it only ever knew a socket. An operator has to hand-edit the host before the link works. Harmless once, tedious across 31 hosts, and a trap for anyone who pastes it as printed. Worth a fix in Eleven: derive the link's origin from a public-host env var when one is set.

1. The architecture default is a real trip hazard. carlos ship and carlos deploy default --target linux-arm64; Eleven's committed binary is lchat-linux-amd64 and its build-linux.sh defaults ARCH=amd64. The two defaults disagree, and nothing catches it until the box tries to exec the binary. Eleven's build matrix should produce arm64 for Carloku.

Step B — the proving box

The infrastructure already exists in the right shape. infra/dev-mesh/ in Eleven's repo is complete tofu for exactly this box — EC2 t4g.small arm64 (commented as "the sizing floor for a node that runs a sidecar"), security group, IAM role and instance profile, EIP, a versioned S3 bucket with a lifecycle rule, and dev.11ch.at delegated from DNSimple into a Route53 zone. ops dev rebuild already destroys and recreates it from nothing. Phase 1 does not design new infrastructure; it ports that shape into https://amadan.net/eleven/infrastructure and points the box at carlos agent instead of lchat router.

Use a fresh namespace, not dev.11ch.at — the existing dev mesh keeps working, and the two must not contend for one zone.

DNS provider. The phase-1 zone is on Route53, so this box runs CARLOS_DNS_PROVIDER=route53. Production is on DNSimple. Since the setting is per box, the two coexist with no conflict — the single-provider constraint bites only within one box.

What phase 1 must prove

Each of these is a review finding converted from an unknown into a rehearsal:

1. Release delivery end to end (the review's C1) — carlos ship Eleven's binary, set channel pointers, and watch a box-side adopt install /opt/carlos/versions/… and mint the live links. This is also where the signing-key migration gets settled: Eleven signs its own releases today (deploy/ops publish, key from Bitwarden), and from here on the authority is Carloku's release key.
2. The manifest shape (C3) — author an app and its manifest as a Carloku operator (carlos instances enable -domain <zone>), then mint instances through the ordinary door. Learn the one-label rule at small scale.
3. The product actually works on CARLOS — log in to home, create a space, invite a member, exchange messages. This is the part no amount of code reading can establish.
4. Sidecars (Open Q7) — provision a bot from nothing. Today's failure is unexplained, and a greenfield box is where the explanation is cheapest to find. If #223 has not landed, this is also where its args field and the <host>-sidecar.env rename get exercised.
5. What of lchat router CARLOS cannot replace. Hooks (in.<host>) and links sit behind the router today, and in.<host> is two labels — which ValidHost refuses. Phase 1 is where their disposition is decided, with nothing depending on the answer.

Phase 1 is finished when a person can sign in, make a space, talk in it, and watch a bot answer — with carlos the only control plane involved.

Phase 2 — move the production hosts, one at a time

Fresh CARLOS boxes in Eleven's AWS account. The two existing boxes keep serving until they are empty, then retire. No box is ever converted in place, which is what removes the registry collision, the fleet-wide lease drain and the ports 80/443 handover from this plan altogether.

The state does not move, again. Old and new boxes point at the same hibernate bucket and prefix, and HibernatePrefix(prefix, host) is prod/<host> on both sides — verified against the live bucket. A moved host is restored by the new box from the keys lchat already wrote. There is no database copy in this plan, and docs/instance-migration.md's copy step is not needed here.

The lease makes it safe by construction. Both sides use the same litestream lss3.Leaser, per host, at <prefix>/<host>/lock.json, TTL 15 minutes. Only one box can hold a host's lease, so a half-moved host cannot be served twice. What the TTL costs is a bounded per-host wait, scheduled deliberately, rather than the fleet-wide outage that in-place conversion would have caused.

Per-host move

For each of the 31 hosts, in an order you choose (start with the least busy):

1. Create the instance record on the right app for that host's zone (per phase 1's manifest work). One host at a time; there is no bulk import.
2. Let the host sleep, or put it to sleep, so its lease is not held.
3. Drop its route on the old box, so lchat will not wake it again.
4. Flip that host's A record to the new box. A specific record beats the *.11ch.at wildcard, so this moves exactly one host and nothing else.
5. Wake and verify — the new box restores from prod/<host>/ and serves. Allow for the lease TTL if the old box held it recently.
6. Rollback: flip the A record back and restore the old box's route. The bucket state is shared, so nothing is stranded.

The wildcard moves last, once every host has an explicit record, and the old boxes come out of DNS entirely before they are destroyed.

Two box-side publishers retire with the old boxes (phase 5): deploy/status-publish.sh generates fleet.json every five minutes, and deploy/ssh-ledger-collect.sh publishes the SSH access ledger on its own timer. Both stop when the boxes are destroyed, and the public status page decays silently rather than loudly. Resolve their rehoming before the last box is destroyed, not after.

The sibling services move too. Hooks and links are router-fronted on the production boxes, so retiring lchat router retires them. They are part of phase 2's surface — not, as an earlier draft claimed, out of scope — and in.<host> needs the disposition phase 1 decided.

What phase 2 still carries

Fresh boxes dissolve the registry collision (C2), the ports 80/443 handover and its reboot race, and the fleet-wide lease drain. What remains real:

• Release delivery must work on the new boxes (proven in phase 1).
• Manifests and 31 instance records still have to be authored.
• CARLOS_POOLS stays unset. inScope gates pool first (reconcile.go:212) and an empty Pools serves only the "" pool (reconcile.go:1006-1011) — correct only while every Eleven manifest carries Pool: "". Pin it.
• Do not pass --region to instances create. App placement overrides the request body (apiinstances.go:558-561), so the flag does nothing here.

Phase 3 — sidecars

Precondition: PR #223 merged and deployed to Carloku. It is 55 commits and +15.7k lines and has not moved since 2026-08-18. It does not go onto the critical path of phases 1-2 (D2).

Why sidecars will not spawn AFTER the migration — and why that is a different question from why they do not run today. An earlier draft claimed the sidecar run hardcode explained Paul's "sidecars don't spin up today". That was wrong, and the review caught it: today's boxes run Eleven's own activator, which spawns bot run from /data/<host>-bot.env and is a working mechanism (messenger activator.go:19-27,243). Whatever stops sidecars today is not this, and remains unexplained — establish it on a live box rather than carrying a guess forward.

What the hardcode does explain is a post-migration failure. CARLOS's exec backend spawns <release binary> sidecar run (backend_exec.go:826) when /data/<host>-sidecar.env exists, while Eleven's binary dispatches bot (main.go:190) — so after the flip the child would exit on a usage error. PR #223 closes it: a sidecar record carries a required args argv field, so carlos sidecars enable -app <app> -args 'bot run' names the command instead of hardcoding it.

There is a second, quieter mismatch on the same path. CARLOS reads <DataDir>/<host>-sidecar.env — its own comment calls this "Eleven's <host>-bot.env, renamed" (backend_exec.go:798). Eleven's existing staged env files are therefore invisible to CARLOS even after #223's args fix. Phase 3 needs an explicit restage or rename step for every sidecar env file — and phase 1 is where that is discovered rather than debugged in production.

Steps. Provision the third box in account 427880590669 via tofu; attach it as a fleet box with CARLOS_INBOUND=none; enrol the bot sidecars with -args 'bot run' and -placement no-inbound. #223 refuses to mint routed instances on a no-inbound box, so the isolation is platform-enforced rather than conventional.

The hooks sidecar does not belong here. lchat hooks run serves in.<host> behind the router — it is inbound. #223's sidecar class is defined as "a serving class that nothing can dial", so modelling it as a #223 sidecar would make it permanently unreachable. But it cannot simply become an ordinary instance with a route either: in.<host> is two labels under its zone and ValidHost accepts only one. Its disposition is decided in phase 1, where nothing depends on the answer.

Phase 4 — edge-fronted ingress (deferred, gated)

The eventual goal is members reaching a nearby edge. It is deferred behind two entry conditions, both of which must be written down as met before any DNS moves:

1. Fleet-placed routes can be steered. They cannot today. internal/steering/steering.go:313 excludes non-shared-pool routes defensively — "the door refuses them" — and the CLI says steering … latency|off (shared pool only, v1). Steered names must also be in PLATFORM_APPS_DOMAIN's zone; a custom domain reaches steering by CNAME to edge.<apps-domain>.
2. The WebSocket hop is measured against direct-serve on a real transatlantic path. Eleven is WebSocket fan-out (hub.go). Nearest-edge wins the TLS and TCP handshake once; thereafter every message pays edge → tunnel → box for the life of the connection. With boxes in Germany/UK and edges in IE/US/UK/AU, that is plausibly a net loss in steady state. Measure it; do not assume it either way.

Two things this phase is not:

• It is not Eleven's multi-edge. docs/multi-edge.md's north star is "a space live on several edges at once", built on the append-only-log-is-a-CRDT property. CARLOS routes to the one box holding the lease; it does not replicate instance state and should not learn to. Replicated spaces stay Eleven's.
• It is not delivered by instances set-upstreams, which is health-probed failover, not geo-routing.

Direct-serve remains the rollback path in every case.

Phase 5 — the public record: status site and prompt registry

Both live in one S3 bucket today (eleven-status-427880590669, served at status.elevenmessenger.com via CloudFront), but measurement shows they are two unrelated problems:

| | Objects | Bytes | |---|---|---| | Whole bucket | 521 | 283,396,762 (~270 MB) | | prompts/ alone | 422 | 282,761,414 (~270 MB) | | Everything else | 99 | ~620 KB |

99.8% of the bucket is the prompt corpus. The status site proper is 620 KB.

The status site → a CARLOS static app

620 KB across 99 objects is a straightforward static app, and carlos deploy serves static natively. But hosting is the easy half. docs/status.md describes two kinds of content and only one is static:

• Committed data — carbon.json, economics.json, contributors.json, music-*.json, ssh/*.json. Authored, reviewed, pushed with ops pages push. Moves cleanly to a static app; ops pages push retires with it.
• Generated data — fleet.json and friends, produced every five minutes by deploy/status-publish.sh on the router box: instance counts, awake vs hibernating, version distribution, shared-service health, replication health.

The generator is the dependency, and it is a phase 2 problem. That script runs on a box phase 2 retires, so the status page starts decaying the moment the last host crosses unless the generator has somewhere to go. Sequence this accordingly: the static half can move early, but the generated half must be resolved before the old boxes are destroyed.

The generator becomes a scheduled CARLOS app (Paul, 2026-08-30). carlos schedule set --app <a> --name fleet --every 30m --path /generate declares it, and a box's runner "wakes a hibernating instance when its tick is due" — so a sleeping generator is the supported path, not a trick. --every has a 1-minute floor, so any cadence under discussion is legal.

Amended 2026-09-05: the cadence argument this spec carried is void. The original text said "use 30 minutes, not 5 — because at 5 it never sleeps", reasoning that a 5-minute tick resets lastActive three times per 15-minute idle window and pins the instance awake. That was true when written and is not true now. Scheduled ticks must not hold an instance awake (#371, on main 2026-08-31) added Service.ServeScheduled and beginScheduledRequest (activator.go:1261), which wakes a sleeping instance and holds the in-flight count for the tick's duration but releases without stamping lastActive; activatorDeliverer.Deliver calls it instead of ServeHTTP (schedule_wiring.go:349). A tick now buys no residency at all.

So pick the cadence from what the page needs, not from the idle window. The status page's own contract is a five-minute fleet.json, and five minutes is now affordable: the instance wakes, generates, and sleeps again one idle window after the last real request. Match the existing cadence unless something else argues against it.

Where it runs still matters, for a smaller reason than before. CARLOS_HIBERNATE_IDLE is still read once from the box environment (activator.go:92) with no per-route or per-app override — gap 3 below stands. But #371 removes the sharp edge that made placement urgent: the generator no longer needs a short idle to sleep, so it no longer wants a box configured against its neighbours. Prefer the shared Carloku fleet anyway — a generator reading the console API needs no fleet-local knowledge, it keeps Eleven's boxes for Eleven's members, and it is the cheaper rung — but this is now a preference, not a constraint.

Worth exploring rather than assuming: CARLOS already knows much of what fleet.json reports — instances, awake state, versions, heartbeats are all console-API facts. Part of the page may become an API consumer rather than a box-side generator, which would remove the box dependency entirely. What CARLOS does not know is Eleven's own aggregate vocabulary (allocation band words, fleet-backup replication health), so some generator is likely to survive; the question is what it runs on and what it reads.

The privacy rules are non-negotiable and must survive the move. docs/status.md states them as holding by construction in the generator — aggregates only, never a hostname. A rehomed generator must carry the same boundary, and the existing contract test (test/status-contract.test.mjs) is the thing that proves it. Port the test with the generator, or the guarantee becomes a claim.

The prompt registry → carlos ledger, not git

Amadan keeps its prompt registry in git because it is small. Eleven's is 270 MB and grows per session, per contributor, in a versioned bucket — Eleven's own docs say why it is not committed: "far too large to commit." Git would store every revision forever and every clone would pull all of it. Three orders of magnitude separate the two cases; the same answer does not fit both.

carlos ledger already provides the right primitives, and per the open-ledgers work it was designed from Eleven's redaction rules in the first place:

• blob — content-addressed, create-only uploads (re-uploading identical bytes is a no-op), which is exactly the corpus's shape.
• append — hash-chained entries for the session index.
• publish — the public manifest, with a title.
• verify — walks a published chain over plain HTTPS and re-hashes every entry, needing no credentials. Anyone can check it.

On rewriting for redaction. Git's rewritability looks like an advantage here and is not the property to want: a force-push makes tampering silent, while a hash chain with public verify makes it detectable. Redaction belongs before append — which is what tools/prompt-registry.mjs already does, redacting credentials and annotating each redaction in place. If something slips through, both models require a repair; only the ledger makes that repair visible. Treat immutability as the feature and the pre-append redaction pass as the control.

The hybrid, made concrete — it is the split Eleven already uses, rehomed: prompts.html is committed code and belongs in the static site's repo; the corpus is data and belongs in ledger blobs. They compose, because ledger verify --url <status-host> points at exactly the host the static app serves.

Open for this phase

• How much of the generator collapses into console-API reads once it is a scheduled app. Where it runs is settled (shared fleet, --every 30m); what it still needs a box-side view for is not.
• What one ledger entry is — a session, a batch, a contributor-day — and how a contributor appends without holding broad credentials.
• Whether the music and SSH ledgers follow the same path. Both are accrued public records with the same shape; the SSH ledger additionally has a box-side collector with the same retirement problem as the fleet generator.
• What happens to the existing corpus at cutover: re-uploaded as blobs (content-addressing makes it idempotent) or left in the old bucket behind a redirect.

What stays Eleven's, permanently

• Mesh replication (meshreplicate.go, prod/mesh/). CARLOS deliberately has no mesh prefix — internal/activator/activator.go:117 records the seam.
• The reaper (reaper.go), reclaiming never-claimed zero-member spaces. CARLOS has no equivalent and needs none.
• Self-serve space creation (controld.go, cloud.go). This becomes a console API caller holding a service credential (#220/#227, merged) rather than a local root daemon — but the quota, slug and capacity policy stay Eleven's.
• The entire E2EE layer, which CARLOS never sees and must never need to.

Capability map

| Eleven today | CARLOS | Verdict | |---|---|---| | ops publish — build + sign | carlos ship, release-keygen | replace | | edge\|beta\|stable, ops promote | carlos promote / pipeline / channels | replace | | ops ship sweep, AUTO=pinned | carlos adopt, instances set-channel | replace | | lchat router subdomain routing | edge + registry routes | replace | | certcache + wildcard DNS-01 | edge wildcard manager, DNSimple backend | replace — same design | | activator.go hibernate + lease | CARLOS activator | replace — ported from Eleven | | Bitwarden vault contract | carlos secrets | replace with care — the vault contract is deliberate | | controld / cloud.go | console API + service credential | rehost, keep policy | | reaper.go, meshreplicate.go | — | stays Eleven's | | bot sidecars | PR #223 | phases 1 and 3 | | hooks sidecar (in.<host>) | neither | not a #223 sidecar, and not an instance record either: two labels fails ValidHost. Disposition decided in phase 1. |

Gaps to file against the platform

1. The reconcile dry-run has no operator door. ReconcileConfig.DryRun exists and is documented as design §4.3's backfill mitigation, but nothing outside internal/provision sets it; the only CLI-reachable DryRun is status.DryRun, the unrelated leak check. An operator cannot ask a box "what would you adopt, and what does adopting it let you later delete?"

The fresh-box strategy (D1) demotes this from blocking to wanted. A box that boots with an empty registry and mints its own records is the ordinary path, not a backfill — the dangerous case was adopting a live box's pre-existing lchat rows, and no box is converted in place any more. Still worth building: the question is the right one to be able to ask, and the next customer arriving with an existing estate will be a genuine backfill.

1. No bulk instance-record import. Phase 2 is 31 hand-authored instances create calls. Tolerable once at this size; it is a real gap the moment a second customer arrives with an existing estate, and it is worth filing now while the pain is concrete.

1. No per-app hibernation idle. CARLOS_HIBERNATE_IDLE is box-wide (activator.go:92) with no route- or app-level override. Scheduled-work apps want a short idle (wake, work, sleep); interactive apps want a long one (do not make a returning member pay a cold wake). One box cannot serve both well today. Amended 2026-09-05: this is no longer urgent. #371 means a tick does not stamp lastActive, so a scheduled app sleeps on the box's own idle without needing one of its own — which was the case that made this gap bite. It stays a real gap for apps whose own traffic wants a different idle from their neighbours', but phase 5 no longer depends on it. Still worth a Route.Idle override.

1. Fleet steering. Blocks phase 4. Latency steering is shared-pool only in v1; fleet-placed routes are refused at the door and excluded by the converger.
2. No "list an account's apps" verb. carlos instances list -account X answers only "account X has no apps"; there is no listing for an account that has apps without naming one. This is a product gap under AGENTS.md's API-and-CLI rule, found while establishing the starting state.
3. ~~infra/eleven-fleet.md is wrong.~~ Closed 2026-09-05: the file no longer exists. It was deleted from main in the open-source redaction pass (2adc71f, 2026-08-27) — two days before this spec was written, so the spec's "Supersedes" line described a file that was already gone. Nothing to rewrite; this document is the only account of Eleven's starting state.
4. Multi-provider wildcards. One CARLOS_DNS_PROVIDER per box. Not blocking today (all four zones are DNSimple), but it is a latent constraint on any directly-serving box whose zones are split.

Open questions

The two-phase reframe settled most of the previous list — the registry disposition, the lease-drain window and the front-door handover are all gone with in-place conversion. What genuinely remains:

1. Which of the 31 hosts belongs to which app, and therefore how many apps there are. Needs an inventory before phase 2 starts; phase 1 establishes the shape at four hosts first.

1. The disposition of in.<host> (hooks). Two labels under the zone, which ValidHost can never accept — so it is neither a #223 sidecar nor an instance record. Decided in phase 1.

1. The signing-key migration. Whether Eleven's release key is retired at cutover or cross-published, and what happens to releases already signed with it. Phase 1 settles it in practice.

1. Why do sidecars not spin up today? Eleven's own activator has a working bot run path (activator.go:19-27,243), so the previously-assumed explanation is void. Phase 1 answers this from nothing.

1. What does a hibernating bot sidecar persist across wakes? The Go sidecar writes a session.json byte-compatible with the Node one. Whether that lives on local disk, in a bucket, or is re-minted per wake decides whether the sidecar box needs a data bucket of its own.

1. What the old boxes cost to keep running during phase 2, and therefore how long the parallel window can be. Both estates run at once until the last host crosses.

1. Where the estate lives long-term. AWS is the phase-2 answer with Tito covering the bill (D9), but docs/planet-scale.md treats Hetzner as the production plane deliberately, on cost. Worth revisiting once real running costs are visible.

Security finding (out of scope here; file separately)

The vault item "AWS dev account (Eleven free tier) — dev-agent access key" resolves to arn:aws:iam::427880590669:user/dev-agent carrying AdministratorAccess — on the same account that holds every production instance's parked state. The name describes something much smaller than the grant. The contrast is in the same vault: activator-prod is properly scoped and refuses s3:GetBucketLocation. Narrow the grant or rename the item; it is unrelated to this migration but was found by it.

What this spec does not do

• It does not move Eleven's data. Old and new boxes share one hibernate bucket and prefix, and HibernatePrefix produces the same keys on both sides — a moved host is restored in place, not copied.
• It does not convert any existing box. The two production boxes serve until empty, then retire.
• It does not change any space's address. Phase 1 runs on a throwaway namespace precisely because an address is an identity, and phase 2 moves hosts without renaming them.
• It does not build replicated spaces. That stays Eleven's mesh work.
• It does not change Eleven's E2EE model in any way.