eleven / messenger Public

docs: the secrets vault — sealed env files in a private off-forge git repo (#768)

Documents the storage/recovery scheme the inventory (docs/secrets.md) was
missing: one SOPS+age-sealed env file per service in a tiny git repo that is
the copy of record, with the age private key as the single root of trust.
States the boundaries (release/Sparkle signing keys stay out, permanently),
the hosting stance (a bare repo on operator infrastructure — no GitHub, no
third-party forge; every clone is a backup), and the explicit rationale for
keeping the repo private even though every value is sealed.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Paul Campbell 83289825f6f879092b11695116af7dba5153cab8 parent 6c582f5
2 files changed, +120 −1
  • docs/secrets-vault.md +117 −0
  • docs/secrets.md +3 −1
diff --git a/docs/secrets-vault.md b/docs/secrets-vault.md
new file mode 100644
index 0000000..5d11bcc
--- /dev/null
+++ b/docs/secrets-vault.md
@@ -0,0 +1,117 @@
+# 🤖 The secrets vault
+
+Where the operator's server-side secrets are stored and recovered from.
+`docs/secrets.md` is the inventory — *what* each secret authorizes, its blast
+radius, how to rotate it; this doc is the storage scheme. Per the inventory's
+rule, operator specifics (the actual remote, box paths, key locations) live in
+`CLAUDE.local.md`, not here.
+
+## The scheme: sealed env files in a small git repo
+
+The vault is a tiny private git repository holding one file per service —
+mirroring the env files the services actually run from — with every value
+sealed by [SOPS](https://github.com/getsops/sops) using an
+[age](https://age-encryption.org) recipient. The vault is the **copy of
+record**: the env files deployed on a box are decryptions of it. The editing
+flow is `sops <file>` (decrypts into your editor, re-seals on save), commit,
+copy the decrypted result to the box, restart the service.
+
+Why this shape and not a secrets-manager service (Vault/OpenBao/cloud KMS):
+
+- **No new always-on service.** A secrets daemon is itself a security-critical
+ process with its own unseal problem, and it puts every service's boot behind
+ its availability. Env files on disk have no failure modes; the vault only
+ changes where they're *kept*, not how they're *consumed*.
+- **Two single-binary tools + git.** `sops` and `age` are static binaries; the
+ whole flow is readable in one sitting and operable by hand — the second
+ rule (built to outlive AI) applied to operations.
+- **Versioned and diffable.** SOPS seals values but leaves names readable, so
+ a diff shows *which* secret changed without showing what it changed to.
+ Rotation history, review, and "what did this env look like last month" come
+ free from git. This replaces hand-maintained password-manager items as the
+ recovery path — the drift where a shipped secret never made it into the
+ backup story is what this design exists to end.
+
+Key material: one age keypair. The **public** (recipient) key is not a secret
+and is committed in the vault's `.sops.yaml`. The **private** key is the root
+of trust: it never enters the vault, lives in the operator's password manager
+plus operator machines, and is the one thing the vault's security reduces to.
+
+## What belongs in it — and what never does
+
+**In:** every secret from `docs/secrets.md` whose rotation story is "swap the
+env and restart" — instance admin tokens, relay tokens, Stripe keys, OAuth
+client secrets, provisioner tokens, the DNS-01 token, sidecar Anthropic keys.
+
+**Never:**
+
+- **The release signing key and the Sparkle key.** The inventory keeps these
+ offline, used only at publish time, and the vault must not quietly undo
+ that: aggregating them here would mean a single age-key compromise could
+ become a ship-code-to-the-fleet compromise. They stay out, permanently.
+- **The age private key itself** — the vault can't hold its own root of trust.
+- **Human credentials** — passkeys, SSH keys, sign-in sessions. Those belong
+ in a password manager and its SSH agent, which keep private keys off disk;
+ a file vault would be a downgrade for them.
+
+## Where it lives
+
+Not on GitHub, and not on any third-party forge. Git is distributed — the
+vault needs no host at all: the remote is a bare repository on
+operator-controlled infrastructure, reached over SSH, and **every clone is a
+complete backup**. The working copies on operator machines plus a periodic
+`git bundle` parked in off-site object storage are the redundancy; the forge
+features a host would add (issues, PRs, CI) are exactly the features a secrets
+repo must never use. Which box, which path, where the bundles land:
+`CLAUDE.local.md`.
+
+Disaster recovery reduces to two things from different places: the age private
+key (password manager) and any clone or bundle of the vault. Together they
+reconstruct every service env; separately each is useless.
+
+## Why the vault is private, even though every value is sealed
+
+The crypto would survive publication — age is sound, and files are sealed *as
+if* the repo were public (that discipline is free, and covers an accidental
+visibility flip). It stays private anyway, deliberately:
+
+- **Publishing is irreversible, so security would collapse to one key,
+ forever.** A public repo is forked, mirrored, and archived within minutes
+ and can never be unpublished. If the age key is *ever* compromised — years
+ later, via a stolen laptop or a bad backup — every version of every secret
+ ever committed becomes readable, and rotation can't recall ciphertext the
+ internet already has. Private, an attacker needs the repo *and* the key:
+ two independent failures instead of one.
+- **Mistake tolerance would drop to zero.** The realistic failure isn't broken
+ crypto; it's a plaintext slip — a file committed before sealing, a
+ `.sops.yaml` rule that misses a key. Private, that's a calm rotation.
+ Public, bots watching the commit firehose for `sk_live_`/`sk-ant-` patterns
+ harvest it in seconds — faster than a revert.
+- **Sealed values still leak the map.** SOPS leaves structure readable by
+ design (that's what makes diffs reviewable): env var names, service
+ filenames, and commit history would advertise which providers we depend on
+ and when keys rotate — and when they don't. Useful diff metadata for us is
+ reconnaissance for an attacker.
+- **Public buys nothing.** The repo's only audience is the operator and their
+ deploy tooling. The product's openness story lives in the product's repos;
+ an ops vault gains no users, contributors, or auditors from being public —
+ there is no benefit on the other side of the scale.
+
+## Bootstrapping
+
+```sh
+age-keygen -o vault.agekey # store the key per CLAUDE.local.md, then shred the file
+git init --bare <remote-path> # on the operator box, over SSH
+# in a fresh clone:
+cat > .sops.yaml <<EOF
+creation_rules:
+ - age: <public key from age-keygen>
+EOF
+sops <service>.env # create/edit — seals on save
+git add -A && git commit -m 'seed' && git push
+```
+
+Deploying a change: `sops -d <service>.env` to a root-only temp file, copy it
+to the box's env path, restart the unit, delete the temp file. Adding a
+secret: seal it here **and** add it to `docs/secrets.md` in the same change —
+the inventory's same-PR rule is what keeps the two in step.
diff --git a/docs/secrets.md b/docs/secrets.md
index a9cd97c..16288fd 100644
--- a/docs/secrets.md
+++ b/docs/secrets.md
@@ -14,7 +14,9 @@ identity keys and the Anthropic key, held by sidecars — machines that are
deliberately *members*, never the server.)
Operator-specific locations (which box, which env file, which 1Password item)
-live in `CLAUDE.local.md` and 1Password, not here.
+live in `CLAUDE.local.md` and 1Password, not here. The storage and recovery
+scheme for these secrets — the sealed vault repo, and why it is deliberately
+private — is `docs/secrets-vault.md`.
## Per-instance