🤖 Managed AI on provisioned spaces — design

Status: design agreed 2026-07-14 (decisions below are Paul's). Built so far: phase 0 (#806 fix + Go invite mint, PRs #864/#865), phase 1 (the usage pipe, PR #867), phase 2 (attach-bot + the onboarding opt-in, #858). Phases 3–5 are unbuilt. This is the plan for making the onboarding "enable AI" answer do something on self-serve spaces: provision a sidecar, meter what it spends, bill it honestly, and eventually move the whole bot trust plane off the serving box.

Related: docs/bots.md (trust model), docs/member-bots.md, docs/cloud-provisioner.md (the attach-bot gap this closes), docs/planet-scale.md (co-spawn + the sidecar invariant), docs/tier-economics.md ("AI is never bundled"), issues #562 (pricing), #668 (usage metering pipe), #685 (evict the trust plane), #806 (sidecar self-provisioning bug).

The gap today

The onboarding toggles are real gates — bots_enabled 403s new-agent creation (main.go) and member-bot charters (memberbots.go); ai_enabled is a genuine kill-switch the sidecar honors on every reconnect (internal/bot/brain.go, via GET /api/bot-config). But a freshly provisioned <sqid>.11ch.at space has no sidecar and no model key, so answering "yes, enable AI" produces nothing: Eleven never appears. The provisioner chain (cloud.go → controld.go → addInstance) contains zero bot hooks.

Meanwhile most of the machinery already exists:

• lchat add-bot (bot.go) automates invite minting, the -bot.env file, ownership, and the unit — or defers to co-spawn on hibernating hosts.
• The activator co-spawns sidecars with their instance and makes sidecar traffic activity-invisible (internal/bot/sidecarmark.go), so a managed sidecar can't hold a space awake or wake it.
• The storage meter (meter.go) is a proven per-space measure → record → enforce loop keyed on the cloud ledger.
• Per-bot model choice already ships (bot_configs.model, admin UI menu, re-read on reconnect); charters carry a per-task override.

Decisions (Paul, 2026-07-14)

1. Eleven only is provisioned for cloud spaces. Admins can add further agents through the existing admin "Add an agent" path; on managed spaces that path must feed the managed sidecar automatically (see Roster follows the admin below).
2. Bedrock is phased, not first: phases 0–3 ship on the existing direct Anthropic key; Bedrock lands with the off-box VPC tier where IAM roles and VPC endpoints actually pay off.
3. Small included allowance funded from a capped pool — free AI is a bounded trial, never an unbounded subsidy.
4. On-box first, then iterate — sidecars stay co-located with instances (today's trust posture) until the bot-host tier exists.
5. Bring-your-own-key is an advanced admin option — paste-a-key first, "click to connect" via an OAuth-style flow when a provider supports one.

Trust model and disclosure

Managed AI means we run a machine that holds the bot's private key and can decrypt exactly the threads the bot is in. That is already the disclosed exception-by-membership in docs/bots.md and already true for the hand-provisioned fleet — but self-serve opt-in turns it into a consumer-facing promise, so:

• AI is opt-out by default. The setup flow writes ai_enabled=false unless the admin takes a specific opt-in step. (Deterministic bots — the command floor, no model, nothing sent anywhere — can stay default-on.)
• The opt-in copy discloses plainly, in the lock-and-key voice: the helper's key lives on Eleven's machines, and it can read the chats it's invited to — that's what makes it able to help. Membership stays visible in the roster.
• The invariants hold throughout: bot code never runs in the server process, the sidecar stays a pure outbound client, and the instance server never stores a usable model key (load-bearing for BYO key, below).

Phase 0 — zero-touch provisioning that works

Prerequisites for everything else; both PR-sized.

• Fix #806: bot run must redeem the invite when a session path is set, the file is absent, and the matching invite env var is present (option 1 in the issue — it also covers the co-spawn path, and it is what makes keys born on the host that runs the bot, which phase 4 relies on).
• Mint agent invites in Go. add-bot shells out to agent/mint-invite.mjs, dragging Node onto any box that provisions. Port the mint to the control binary.

Phase 1 — meter before money (#668)

Measure first, price from data. No billing in this phase.

• Parse the usage block in internal/bot/anthropic.go (today anthResponse doesn't even deserialize it). Record all four counters — input, output, cache-write, cache-read — they're priced differently.
• Aggregate in the sidecar per (day, model, dimension); flush to a new cloud POST /api/usage authed by a per-instance meter token minted at create time (mint/list/revoke pattern copied from the push relay). Ledger: usage(space_id, day, dimension, quantity) beside storage_bytes.
• Add bot-activity dimensions (replies, charter runs) on the same pipe.
• Run silently on the existing sidecar fleet for a couple of weeks; the allowance and add-on price in phase 3 are set from this data.

The meter flush doubles as the budget heartbeat: the response to POST /api/usage carries the space's current AI budget state, so no new cloud→instance channel is needed for enforcement (phase 3).

Phase 2 — the opt-in that provisions (on-box)

• Onboarding: the extras step's AI question becomes an explicit opt-in chip with the disclosure copy; declining (or skipping) writes ai_enabled=false. Opting in triggers provisioning during setup — "Eleven is joining your space…" — and Eleven's welcome DM lands seconds later. The same opt-in surfaces in the Admin app for existing spaces (note: the Admin mini-app's setFeatures bridge is currently dead code — wire it up here).
• Signal path: instance → cloud over the meter-token channel ("this space wants AI") → cloud calls a new controld POST /instances/{host}/bot, which runs the add-bot logic: mint the Eleven invite, stage /data/<host>-bot.env with the fleet ANTHROPIC_API_KEY, chown. On a hibernating host that's all it does — co-spawn picks the sidecar up (the activator needs one small hook: notice a newly staged -bot.env while the child is already awake, so opt-in doesn't wait for the next sleep/wake cycle).
• Roster follows the admin: when an admin adds another agent on a managed space, the same signal path appends that identity's invite to the bot env and restarts the sidecar; post-#806 it self-provisions. The sidecar's fixed two-identity env shape (LCHAT_INVITE / LCHAT_ELEVEN_INVITE) generalizes to a numbered or manifest form as part of this.
• Latency budget: chip-tap to welcome DM in seconds, not sweeps — this is why the signal is a push over the meter channel, not a stats-poll.

Phase 3 — usage accrues to the subscription

• Stripe Billing Meters on the existing per-space subscription (billing.go — the provisioner's Stripe integration, not the in-space Connect one). Metered price = provider cost × a published markup, recorded in docs/tier-economics.md like every other rate. AI becomes the first real marginal-cost line in the public books.
• Included allowance from a capped pool: quantities under the per-space monthly allowance never forward to Stripe; a fleet-wide monthly AI budget (mirror of the creates_log creation-budget circuit breaker) bounds aggregate subsidized spend no matter how many free spaces opt in. Allowance and pool sizes come from phase 1 data. Cost reality check at today's list prices: a typical cached Sonnet reply runs ~1–1.5¢, so ~100 replies/month ≈ €1.2/month against Family's €4.68/year net — the allowance is trial-sized by necessity, and "AI is never bundled" stands.
• Exhaustion behavior: the meter-flush response flips the space to budget-exhausted; the sidecar's existing AIEnabled gate grows that state — the bot drops to the deterministic command floor and says so once, honestly. Per the conduct rule (knowledge.md §20), Eleven describes the upgrade path and never posts a payment link.
• Abuse guards land with billing, not after: a per-space daily reply ceiling independent of budget (minEvery floors charter schedules but nothing bounds conversational replies), and loop protection stays audited.

Phase 4 — the bot-host tier: egress-only VPC (#685)

The "hardened off-box sidecar tier" from #550 Part 4 v2; sidecars follow the commons, not the provider.

• Sandbox: systemd, not Docker. The bot unit already carries near-maximal hardening (empty capability set, ProtectSystem=strict, MemoryDenyWriteExecute — possible because the sidecar is pure Go — syscall filter, MemoryMax), and the sidecar runs no untrusted code: member-app code executes in the client's sandboxed iframe, charters are bounded tool loops. Docker would add a daemon and an image pipeline — against the one-boring-binary rule — for less isolation than the unit provides. The real gap is that co-spawned sidecars inherit the router's sandbox; moving them to a dedicated host puts each space back under its own hardened unit (optionally per-space DynamicUser).
• Host shape: EC2 (Graviton, instance-role pattern from infra/dev/compute.tf) in a private subnet — no public IP, no inbound anything; egress via NAT restricted to 443. The sidecar is already a pure outbound client; the X-Lchat-Sidecar marker means off-box sidecars can't wake or hold instances awake, and charter due-time wakes already work from off-box (POST /api/charter-next-due). The IaC boundary test holds: tofu builds the box and role; the lchat binary (a bot-host mode) learns which spaces it hosts by outbound-polling the control plane.
• Keys are born on the bot host. Provisioning hands the host one-time invites; the sidecar self-provisions (#806 fix), so bot private keys never transit anywhere. Combined with #685 this evicts bot keys and the model key from the serving box.
• Lifecycle v1: off-box sidecars run always-on (tens of MB each; one small host carries dozens; scale horizontally before getting clever).
• Ops debt that must close before this scales: #811 — awake bot-sidecar instances never pick up auto-updates.

Phase 5 — the provider seam: Bedrock, local models, catalog

One small interface in internal/bot (today's client becomes implementation #1):

1. Anthropic direct — the existing hand-rolled Messages client; stays the default for BYO key and self-hosters (ANTHROPIC_BASE_URL seam exists).
2. Bedrock — for the managed VPC tier. The draw is not model choice: it's IAM-role auth (no ANTHROPIC_API_KEY exists anywhere for managed spaces — the shared-key blast radius disappears) and VPC interface endpoints (model traffic never touches the public internet). Same Messages body under SigV4 — home/sigv4.go proves the no-dependency signer pattern; model ids gain the anthropic. prefix. Check eu-west-1 / EU cross-region profile availability — EU-resident inference is a real selling point given bots see plaintext.
3. OpenAI-compatible local — the SLM companion direction: a small model in the same private network, called by the sidecar. First workload to move: the addressing classifier (hardcoded Haiku today, brain.go) — highest volume, lowest difficulty; moving it in-network cuts both spend and plaintext egress.

Model management: per-bot choice already exists; the improvement is a fleet-served model catalog (menu entries + ids) so admin.js stops hardcoding model ids and fleet-wide swaps/deprecations need no client redeploy. Keep the menu small: default, better, cheaper.

Bring your own key (advanced admin option)

• The invariant shapes the design: compromising the server must not yield the model key (CLAUDE.md), so a BYO key is never persisted in the instance DB. Set flow: admin UI → instance (held in memory only, never stored, never logged) → cloud over the meter channel → controld writes it into /data/<host>-bot.env → sidecar restart. Same at-rest posture as the fleet key today; treated like the admin token in transit.
• BYO spaces don't meter and aren't billed for AI (#668's rule: their key, their cost). The allowance/budget machinery disarms; ai_enabled still works as the kill switch.
• Click to connect: Anthropic offers no third-party OAuth key provisioning today, so v1 is paste-a-key (Anthropic key, Anthropic wire format). The connect broker (lchat-connect, which already holds OAuth client pairs for GitHub/DNSimple) is the natural seam when a provider supports key issuance by consent — OpenRouter's PKCE flow exists today if we ever want it, with its own disclosure (a third party would see the bot's plaintext); an Anthropic OAuth flow is a watch item.
• Advanced means advanced: buried in the admin console, not in onboarding.
• v2 candidates: per-space Bedrock via cross-account role (Business tier), arbitrary OpenAI-compatible base URL + key for self-managed local models.

Kill switches and guards (summary)

• Per-space: ai_enabled (exists, enforced in the sidecar), budget exhaustion (phase 3), daily reply ceiling (phase 3).
• Fleet: the subsidy-pool circuit breaker (phase 3); env kill on bot hosts.
• Egress: charter/research URL fetches from our infrastructure need an egress policy on the bot host (a prompt-injected charter fetching internal endpoints is SSRF from our VPC) — the serving box's unfurl SSRF guard has no sidecar equivalent today.
• Terms: one pass over Anthropic/Bedrock commercial terms (resale with markup, data-retention posture we advertise) before the price goes public.

Sequencing

Phase 0 → 1 → 2 ship independently and in order; 3 needs 1's data and 2's opt-in; 4 and 5 ride the #685 arc and don't block 2–3. Native parity is mostly free (onboarding/admin render through the webview) but the opt-in flow gets an explicit iOS verify. docs/tier-economics.md gains the AI rate line when phase 3 prices are set, and its stale "a bot-sidecar space never sleeps" note gets reconciled with co-spawn (#670) and the observed #811 behavior.