server+web: incoming webhooks — in.<host> ingest sidecar, admin registry, Webhooks app (#610)
The webhook-ingest sidecar (lchat hooks run, internal/hooks) is the bot sidecar's inverse: it accepts inbound HTTP on in.<host> (a plain registry route, kind 'hooks' — no router code changes; per-host autocert covers TLS) and turns each request into an encrypted webhooks.<ns> event posted into one thread. Strictly write-only: no websocket, no message reads, no instance key — definitions are sealed to its service identity, and per webhook it holds only that webhook's keypair + token, wrapped to exactly one thread's key. - hooks.go + db.go: webhooks table (opaque def/meta blobs) + admin-gated create/list/remove, service-identity discovery, sidecar defs poll + deploy ack (both narrowed to the recorded service identity). - hookscmd.go: lchat hooks run (socket behind the router, --addr for dev) and lchat add-hooks <host> (service invite via hooks_service, env file, in.<host> route, lchat-hooks@ unit — the add-bot shape); deploy/update and remove now carry the hooks sidecar along. - web: admin-only Webhooks app (create with name + event namespace + target group-or-space-thread, list with instant URLs + Deploying…/Active state, remove) riding the setup bridge; creation mints the webhook's keypair and seals its credentials to the sidecar client-side. - Space thread: a webhook posts there via the explicit agent membership row + admin-wrapped key — the deliberate opt-in threadMember was built for. - Contract: webhooks.* types are unclaimed by design; vectors pin the ignore-unclaimed rule on web (real foldEvent/isAppType gate) and native (MessagePayload.isRenderableChat), test/event-vectors/webhooks.json. - Docs: docs/webhooks.md (design + trust model), instance-router registry note, planet-scale guide census/reference entries, changelog. Supersedes the /hooks/* path-split approach of PR #617.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DdpyYWMyEhubwEPEWNmJZG
23 files changed,
+2472
−21
CLAUDE.md+8 −0changelog/2026-07-17-incoming-webhooks.md+1 −0control.go+18 −2db.go+142 −0deploy/systemd/lchat-hooks@.service+55 −0docs/instance-router.md+11 −0docs/planet-scale-guide/processes.html+1 −0docs/planet-scale-guide/reference.html+3 −0docs/webhooks.md+174 −0hooks.go+310 −0hooks_test.go+425 −0hookscmd.go+218 −0internal/hooks/hooks.go+471 −0ios/LChatCore/Tests/LChatCoreTests/Fixtures/webhooks-vector.json+44 −0ios/LChatCore/Tests/LChatCoreTests/WebhooksVectorTests.swift+40 −0main.go+39 −2test/event-vectors.test.mjs+20 −0test/event-vectors/index.js+2 −0test/event-vectors/webhooks.json+59 −0updater_test.go+7 −6web/static/apps/webhooks.js+297 −0web/static/newchat.js+18 −11web/static/shell.js+109 −0
diff --git a/CLAUDE.md b/CLAUDE.md| index d677a38..4c2d0a6 100644 |
| --- a/CLAUDE.md |
| +++ b/CLAUDE.md |
| @@ -156,6 +156,14 @@ in, so deploy is one file (no cgo). |
| (`botcmd.go`, `internal/bot/`), no Node runtime. `agent/` is the **Node |
| reference implementation, kept on purpose** (a runnable example + the wire-format |
| illustration the Go tests pin against) — don't delete it or let it rot. |
| +- `hooks.go` — **incoming webhooks** (#610): the admin-managed registry of |
| + webhook definition blobs (opaque to the server) behind the webhook-ingest |
| + sidecar — the bot sidecar's inverse. `lchat hooks run` (`hookscmd.go`, |
| + `internal/hooks/`) serves `in.<host>` behind the router and turns inbound |
| + HTTP into encrypted `webhooks.<ns>` events posted into one thread; it is |
| + strictly write-only (no websocket, no message reads, no instance key). |
| + Stood up per instance by `lchat add-hooks`. Design + trust model: |
| + `docs/webhooks.md`; management UI `web/static/apps/webhooks.js` (admin-only). |
| - `unfurl.go` — OpenGraph link previews (SSRF-guarded fetch + cache). |
| - `linkscmd.go` — `lchat links run`, the `11ch.at` short-link redirector for |
| message permalinks (issue #905): a bare apex request bounces to |
diff --git a/changelog/2026-07-17-incoming-webhooks.md b/changelog/2026-07-17-incoming-webhooks.md| new file mode 100644 |
| index 0000000..e92c3e6 |
| --- /dev/null |
| +++ b/changelog/2026-07-17-incoming-webhooks.md |
| @@ -0,0 +1 @@ |
| +🤖 Admins can now create incoming webhooks: give an outside service (GitHub, Stripe, anything that can send an HTTP request) a unique URL, and everything it sends arrives as an encrypted event in the chat you choose — the server never sees the content, and the receiver can never read anything back. Find it under Apps → Webhooks (admins only), once your space's operator has switched the ingest service on. |
diff --git a/control.go b/control.go| index ea6d658..6d9a397 100644 |
| --- a/control.go |
| +++ b/control.go |
| @@ -575,11 +575,19 @@ func runRemove(args []string) { |
| if err := reg.delete(host); err != nil { |
| die("remove: %v", err) |
| } |
| + if rt.Kind == "instance" { |
| + // An instance's webhook-ingest route (in.<host>, add-hooks) is part of |
| + // the instance: drop it too so the ingest URL dies with the space. |
| + if err := reg.delete("in." + host); err != nil { |
| + die("remove: %v", err) |
| + } |
| + } |
| if err := systemctl("reload", *routerUnit); err != nil { |
| die("remove: reload %s: %v", *routerUnit, err) |
| } |
| if *stop && rt.Kind == "instance" { |
| - _ = systemctl("disable", "--now", *botUnit+host) // best-effort: a sidecar may not exist |
| + _ = systemctl("disable", "--now", *botUnit+host) // best-effort: a sidecar may not exist |
| + _ = systemctl("disable", "--now", "lchat-hooks@"+host) // best-effort: same for webhook ingest |
| if err := systemctl("disable", "--now", *unitTmpl+host); err != nil { |
| fmt.Printf("removed %s (route deleted), but stopping the unit failed: %v\n", host, err) |
| return |
| @@ -593,7 +601,6 @@ func runRemove(args []string) { |
| } |
| } |
| - |
| // canReset reports why host may not be reset, or nil if it may. The canary gate |
| // IS the safety model: reset wipes a live DB, so it refuses anything not |
| // explicitly flagged throwaway (registry.go: Route.Canary) — a real member |
| @@ -1317,6 +1324,7 @@ func deployHostVersion(reg *Registry, rt *Route, unit, ver string) (changed bool |
| return true, fmt.Errorf("record version: %v", err) |
| } |
| restartBotSidecar(host) |
| + restartHooksSidecar(host) |
| return true, nil |
| } |
| @@ -1332,6 +1340,14 @@ func restartBotSidecar(host string) { |
| } |
| } |
| +// restartHooksSidecar: same lock-step contract for the webhook-ingest sidecar |
| +// (deploy/systemd/lchat-hooks@.service execs the live symlink too). |
| +func restartHooksSidecar(host string) { |
| + if err := systemctl("try-restart", "lchat-hooks@"+host); err != nil { |
| + log.Printf("deploy: hooks sidecar restart for %s failed (instance is healthy on the new version): %v", host, err) |
| + } |
| +} |
| + |
| // acquireDeployLock serialises everything that runs the repoint/restart/ |
| // health-check dance — deploy, rollback, and the timer-fired `update run` — |
| // across processes, so an auto-update can never interleave with an operator's |
diff --git a/db.go b/db.go| index e7ebe9f..bccaff5 100644 |
| --- a/db.go |
| +++ b/db.go |
| @@ -678,6 +678,22 @@ CREATE TABLE IF NOT EXISTS member_bots ( |
| created_at INTEGER NOT NULL, |
| updated_at INTEGER NOT NULL |
| ); |
| +-- Incoming webhooks (docs/webhooks.md, #610): each row is one webhook — an |
| +-- agent identity that turns inbound HTTP into encrypted typed events. Both |
| +-- blobs are opaque here: def_blob is sealed to the hooks sidecar's service |
| +-- identity (it alone can read the webhook's private key, member token, and |
| +-- URL secret); meta_blob is instance-key-encrypted (the admin app's list |
| +-- view). The server learns only (id, which agent user, sizes, times) — never |
| +-- a secret, a key, or which external service feeds it. deployed_at is the |
| +-- sidecar's plaintext "route is live" ack, 0 until the first poll picks it up. |
| +CREATE TABLE IF NOT EXISTS webhooks ( |
| + id TEXT PRIMARY KEY, |
| + user_id TEXT NOT NULL, |
| + def_blob TEXT NOT NULL DEFAULT '', |
| + meta_blob TEXT NOT NULL DEFAULT '', |
| + deployed_at INTEGER NOT NULL DEFAULT 0, |
| + created_at INTEGER NOT NULL |
| +); |
| -- Tamper evidence (docs/tamper-evidence.md, #573): the append-only event |
| -- journal. The messages table mutates on purpose (edits UPDATE, unsend hard- |
| -- DELETEs), so checkpoints commit to this journal instead: every message |
| @@ -1219,6 +1235,24 @@ func (db *DB) createUser(label string) (string, error) { |
| return id, err |
| } |
| +// createDeviceToken mints a device token for an existing user outside the |
| +// invite-claim path — used for webhook identities (hooks.go), whose creator |
| +// receives the token once and seals it into the sidecar's definition blob. |
| +// Same storage discipline as claimKeyInvite: only the hash is kept. |
| +func (db *DB) createDeviceToken(userID, label string) (string, error) { |
| + tok, err := randToken() |
| + if err != nil { |
| + return "", err |
| + } |
| + _, err = db.sql.Exec( |
| + "INSERT INTO device_tokens(token_hash, user_id, label, created_at) VALUES(?, ?, ?, ?)", |
| + hashToken(tok), userID, label, time.Now().Unix()) |
| + if err != nil { |
| + return "", err |
| + } |
| + return tok, nil |
| +} |
| + |
| func (db *DB) listUsers() ([]User, error) { |
| now := time.Now().Unix() |
| rows, err := db.sql.Query(` |
| @@ -4923,6 +4957,114 @@ func (db *DB) deleteMemberBot(id string) (bool, error) { |
| return n > 0, nil |
| } |
| +// ---- incoming webhooks (hooks.go, docs/webhooks.md) ---------------------- |
| + |
| +// Webhook is one incoming-webhook row. The server can open neither blob: def |
| +// is sealed to the hooks sidecar's service identity, meta to the instance key. |
| +type Webhook struct { |
| + ID string `json:"id"` |
| + UserID string `json:"user_id"` |
| + DefBlob string `json:"def_blob,omitempty"` |
| + MetaBlob string `json:"meta_blob,omitempty"` |
| + DeployedAt int64 `json:"deployed_at"` |
| + CreatedAt int64 `json:"created_at"` |
| +} |
| + |
| +func (db *DB) createWebhook(id, userID string) error { |
| + _, err := db.sql.Exec( |
| + "INSERT INTO webhooks(id, user_id, created_at) VALUES(?, ?, ?)", |
| + id, userID, time.Now().Unix()) |
| + return err |
| +} |
| + |
| +// setWebhookBlobs stores both encrypted blobs (the create flow's second step — |
| +// the definition can only be sealed once the create response has minted the |
| +// webhook's device token). |
| +func (db *DB) setWebhookBlobs(id, defBlob, metaBlob string) (bool, error) { |
| + res, err := db.sql.Exec("UPDATE webhooks SET def_blob = ?, meta_blob = ? WHERE id = ?", defBlob, metaBlob, id) |
| + if err != nil { |
| + return false, err |
| + } |
| + n, _ := res.RowsAffected() |
| + return n > 0, nil |
| +} |
| + |
| +func (db *DB) getWebhook(id string) (Webhook, bool, error) { |
| + var h Webhook |
| + err := db.sql.QueryRow( |
| + "SELECT id, user_id, def_blob, meta_blob, deployed_at, created_at FROM webhooks WHERE id = ?", id, |
| + ).Scan(&h.ID, &h.UserID, &h.DefBlob, &h.MetaBlob, &h.DeployedAt, &h.CreatedAt) |
| + if err == sql.ErrNoRows { |
| + return Webhook{}, false, nil |
| + } |
| + if err != nil { |
| + return Webhook{}, false, err |
| + } |
| + return h, true, nil |
| +} |
| + |
| +// listWebhooks returns every webhook, oldest first, meta blobs included (the |
| +// admin app's list view; def blobs stay out — the console never needs them). |
| +func (db *DB) listWebhooks() ([]Webhook, error) { |
| + rows, err := db.sql.Query( |
| + "SELECT id, user_id, meta_blob, deployed_at, created_at FROM webhooks ORDER BY created_at, id") |
| + if err != nil { |
| + return nil, err |
| + } |
| + defer rows.Close() |
| + out := []Webhook{} |
| + for rows.Next() { |
| + var h Webhook |
| + if err := rows.Scan(&h.ID, &h.UserID, &h.MetaBlob, &h.DeployedAt, &h.CreatedAt); err != nil { |
| + return nil, err |
| + } |
| + out = append(out, h) |
| + } |
| + return out, rows.Err() |
| +} |
| + |
| +// listWebhookDefs returns the rows the hooks sidecar polls: only webhooks whose |
| +// definition has landed (the create flow's second step may still be in flight). |
| +func (db *DB) listWebhookDefs() ([]Webhook, error) { |
| + rows, err := db.sql.Query( |
| + "SELECT id, user_id, def_blob, deployed_at, created_at FROM webhooks WHERE def_blob != '' ORDER BY created_at, id") |
| + if err != nil { |
| + return nil, err |
| + } |
| + defer rows.Close() |
| + out := []Webhook{} |
| + for rows.Next() { |
| + var h Webhook |
| + if err := rows.Scan(&h.ID, &h.UserID, &h.DefBlob, &h.DeployedAt, &h.CreatedAt); err != nil { |
| + return nil, err |
| + } |
| + out = append(out, h) |
| + } |
| + return out, rows.Err() |
| +} |
| + |
| +// setWebhookDeployed records the sidecar's "route is live" ack (first ack wins; |
| +// re-acks on later polls are no-ops so the timestamp stays the first go-live). |
| +func (db *DB) setWebhookDeployed(id string) error { |
| + _, err := db.sql.Exec("UPDATE webhooks SET deployed_at = ? WHERE id = ? AND deployed_at = 0", time.Now().Unix(), id) |
| + return err |
| +} |
| + |
| +func (db *DB) deleteWebhook(id string) (bool, error) { |
| + res, err := db.sql.Exec("DELETE FROM webhooks WHERE id = ?", id) |
| + if err != nil { |
| + return false, err |
| + } |
| + n, _ := res.RowsAffected() |
| + return n > 0, nil |
| +} |
| + |
| +func (db *DB) countWebhooks() (int, error) { |
| + var n int |
| + err := db.sql.QueryRow("SELECT COUNT(*) FROM webhooks").Scan(&n) |
| + return n, err |
| +} |
| + |
| // ---- event journal + checkpoints (tamper evidence, checkpoint.go) -------- |
| // |
| // The journal is append-only: rows are written inside the same transaction as |
diff --git a/deploy/systemd/lchat-hooks@.service b/deploy/systemd/lchat-hooks@.service| new file mode 100644 |
| index 0000000..8c3a374 |
| --- /dev/null |
| +++ b/deploy/systemd/lchat-hooks@.service |
| @@ -0,0 +1,55 @@ |
| +[Unit] |
| +Description=lchat webhook-ingest sidecar for %i |
| +After=network-online.target lchat@%i.service |
| +Wants=network-online.target |
| + |
| +[Service] |
| +# add-hooks writes /data/%i-hooks.env (0600) with LCHAT_BASE, the service |
| +# identity's one-time agent invite, the session path, and the socket path. |
| +EnvironmentFile=-/data/%i-hooks.env |
| +# Same versioned Go binary as the instance (the per-host live symlink), so the |
| +# ingest sidecar's code stays in lock-step with the app by construction — |
| +# exactly the lchat-bot@ arrangement. `hooks run` reads everything from the |
| +# EnvironmentFile above. |
| +ExecStart=/opt/lchat/live/%i hooks run |
| +Restart=always |
| +RestartSec=2 |
| + |
| +# --- hardening (see docs/instance-router.md § Hardening) --- |
| +# Unlike the bot sidecar this process DOES listen — on a unix socket under |
| +# /run/lchat that only the router dials; it still binds no TCP port and needs |
| +# no capabilities. It is the inverse of the bot: inbound requests come in, |
| +# encrypted posts go out, and nothing here can read a conversation. |
| +User=lchat |
| +Group=lchat |
| +CapabilityBoundingSet= |
| +NoNewPrivileges=yes |
| +ProtectSystem=strict |
| +ProtectHome=yes |
| +# Writes its service-identity session under /data/%i-hooks/ and its listening |
| +# socket under /run/lchat. HOME lands in the private /tmp. |
| +ReadWritePaths=/data /run/lchat |
| +Environment=HOME=/tmp |
| +PrivateTmp=yes |
| +PrivateDevices=yes |
| +ProtectClock=yes |
| +ProtectHostname=yes |
| +ProtectKernelTunables=yes |
| +ProtectKernelModules=yes |
| +ProtectKernelLogs=yes |
| +ProtectControlGroups=yes |
| +ProtectProc=invisible |
| +RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX |
| +RestrictNamespaces=yes |
| +RestrictRealtime=yes |
| +RestrictSUIDSGID=yes |
| +LockPersonality=yes |
| +MemoryDenyWriteExecute=yes |
| +SystemCallFilter=@system-service |
| +SystemCallErrorNumber=EPERM |
| +SystemCallArchitectures=native |
| +MemoryMax=256M |
| +TasksMax=256 |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
diff --git a/docs/instance-router.md b/docs/instance-router.md| index 449aef7..5a63869 100644 |
| --- a/docs/instance-router.md |
| +++ b/docs/instance-router.md |
| @@ -329,6 +329,7 @@ used to be env now lives: |
| CREATE TABLE routes ( |
| host TEXT PRIMARY KEY, |
| kind TEXT NOT NULL, -- 'instance' (messenger, lifecycle-managed) | 'service' (home, push, …) |
| + -- | 'hooks' (an instance's in.<host> webhook-ingest sidecar, add-hooks) |
| socket TEXT NOT NULL, -- unix socket the router proxies to |
| status TEXT NOT NULL, -- pending|creating|starting|issuing|ready|failed |
| last_error TEXT, |
| @@ -343,6 +344,16 @@ CREATE TABLE routes ( |
| ); |
| ``` |
| +🤖 The `hooks` kind is how incoming webhooks reach an instance (issue #610, |
| +`docs/webhooks.md`): `lchat add-hooks <host>` registers `in.<host>` → |
| +`/run/lchat/<host>-hooks.sock`, where `lchat-hooks@<host>` (the webhook-ingest |
| +sidecar) listens. No router code special-cases it — it's an ordinary route. |
| +TLS rides the per-host autocert path (the `*.domain` wildcard cert covers one |
| +label only; wildcard DNS resolves two labels down with no zone change), so an |
| +11ch.at space needs no DNS work; a custom-domain space's owner adds an |
| +`in.<host>` record. `lchat remove <host>` deletes the `in.<host>` row (and, |
| +with `--stop`, the hooks unit) along with the instance's. |
| + |
| ### Per-instance config: a per-instance EnvironmentFile |
| Config stays env-shaped — the instance reads the same `LCHAT_*` vars it always |
diff --git a/docs/planet-scale-guide/processes.html b/docs/planet-scale-guide/processes.html| index 0ecfcd3..13c3d79 100644 |
| --- a/docs/planet-scale-guide/processes.html |
| +++ b/docs/planet-scale-guide/processes.html |
| @@ -38,6 +38,7 @@ |
| <tr><td><code>lchat-router.service</code></td><td>The <a href="router.html">router</a>: TLS + routing; hosts the <a href="hibernation.html">activator</a> and <a href="mesh.html">mesh</a> when configured.</td><td><code>/opt/lchat/lchat router --registry /data/registry.db --acme-cache /data/acme</code></td><td><code>lchat</code></td></tr> |
| <tr><td><code>lchat@<host>.service</code></td><td>One chat space instance (template unit; one per host).</td><td><code>/opt/lchat/live/<host> --socket /run/lchat/<host>.sock --db /data/<host>.db</code></td><td><code>lchat</code></td></tr> |
| <tr><td><code>lchat-bot@<host>.service</code></td><td>The optional bot sidecar for one instance (template unit).</td><td><code>/opt/lchat/live/<host> bot run</code></td><td><code>lchat</code></td></tr> |
| + <tr><td><code>lchat-hooks@<host>.service</code></td><td>The optional webhook-ingest sidecar for one instance (template unit): serves <code>in.<host></code> behind the router, turning inbound webhook HTTP into encrypted events. Stood up by <code>lchat add-hooks <host></code>. <span class="status partial">built, not yet deployed</span></td><td><code>/opt/lchat/live/<host> hooks run</code></td><td><code>lchat</code></td></tr> |
| <tr><td><code>lchat-home.service</code></td><td>The home server: the anonymous, passkey-gated backup store (a separate binary from the same repository).</td><td><code>/opt/lchat/home --addr /run/lchat/home.sock --db /data/home.db</code></td><td><code>lchat</code></td></tr> |
| <tr><td><code>lchat-push.service</code></td><td>The push relay: the central Apple push-notification signer (separate binary).</td><td><code>/opt/lchat/push --addr /run/lchat/push.sock --db /data/push.db</code></td><td><code>lchat</code></td></tr> |
| <tr><td><code>lchat-links.service</code></td><td>The <code>11ch.at</code> short-link redirector for message permalinks (issue #905) — unlike home/push, a subcommand of the main <code>lchat</code> binary, not a separate one; holds no database (an in-memory, TTL-bounded verification cache only). <span class="status partial">built, not yet deployed</span></td><td><code>/opt/lchat/lchat links run --socket /run/lchat/links.sock</code></td><td><code>lchat</code></td></tr> |
diff --git a/docs/planet-scale-guide/reference.html b/docs/planet-scale-guide/reference.html| index e36d1ac..88ebf82 100644 |
| --- a/docs/planet-scale-guide/reference.html |
| +++ b/docs/planet-scale-guide/reference.html |
| @@ -35,6 +35,8 @@ |
| <dd>Registers a <code>service</code>-kind route (home server, push relay, cloud provisioner, or any third-party backend) after verifying the socket answers. No unit is created and no version is tracked — the router only forwards. Explained in <a href="processes.html#sibling-services">part 03</a>.</dd> |
| <dt><code>lchat add-bot <host> [--model <model>]</code></dt> |
| <dd>Provisions the bot sidecar for an instance: mints one agent invite per identity, writes <code>/data/<host>-bot.env</code> and the session directory, and enables <code>lchat-bot@<host></code> (on a hibernating host the unit is skipped; the activator co-spawns instead). Reads <code>ANTHROPIC_API_KEY</code> from the calling environment if AI behaviour is wanted. Explained in <a href="processes.html#bot-sidecars">part 03</a>.</dd> |
| + <dt><code>lchat add-hooks <host></code> <span class="status partial">built, not yet deployed</span></dt> |
| + <dd>Provisions the webhook-ingest sidecar for an instance (incoming webhooks, issue #610): mints the service identity’s agent invite, writes <code>/data/<host>-hooks.env</code> and the session directory, registers the <code>in.<host></code> registry route (kind <code>hooks</code>) pointing at <code>/run/lchat/<host>-hooks.sock</code>, and enables <code>lchat-hooks@<host></code>. Admins then create webhooks from the app’s Webhooks panel. Design and trust model: <code>docs/webhooks.md</code>.</dd> |
| <dt><code>lchat remove <host> [--stop]</code></dt> |
| <dd>Deletes the host’s registry row and reloads the router (traffic stops); with <code>--stop</code> also stops and disables the instance and sidecar units. <strong>Never touches data.</strong> Explained in <a href="spaces.html#removing">part 05</a>.</dd> |
| <dt><code>lchat hibernate <host> --on | --off</code></dt> |
| @@ -184,6 +186,7 @@ |
| <tr><td><code>lchat-router.service</code></td><td>The router (+ activator + mesh)</td><td><code>bootstrap</code></td><td><code>journalctl -u lchat-router</code></td></tr> |
| <tr><td><code>lchat@<host>.service</code></td><td>One instance</td><td><code>add-instance</code></td><td><code>journalctl -u 'lchat@<host>'</code></td></tr> |
| <tr><td><code>lchat-bot@<host>.service</code></td><td>One bot sidecar</td><td><code>add-bot</code></td><td><code>journalctl -u 'lchat-bot@<host>'</code></td></tr> |
| + <tr><td><code>lchat-hooks@<host>.service</code></td><td>One webhook-ingest sidecar</td><td><code>add-hooks</code></td><td><code>journalctl -u 'lchat-hooks@<host>'</code></td></tr> |
| <tr><td><code>lchat-home.service</code> / <code>lchat-push.service</code></td><td>Home server / push relay</td><td>manually, once</td><td><code>journalctl -u lchat-home</code> etc.</td></tr> |
| <tr><td><code>lchat-control.service</code></td><td>Control daemon (root)</td><td>manually, once</td><td><code>journalctl -u lchat-control</code></td></tr> |
| <tr><td><code>lchat-cloud.service</code></td><td>Cloud provisioner</td><td>manually, once</td><td><code>journalctl -u lchat-cloud</code></td></tr> |
diff --git a/docs/webhooks.md b/docs/webhooks.md| new file mode 100644 |
| index 0000000..0033c5e |
| --- /dev/null |
| +++ b/docs/webhooks.md |
| @@ -0,0 +1,174 @@ |
| +# 🤖 Incoming webhooks: the ingest sidecar (`lchat hooks run`) |
| + |
| +Issue #610. External services (GitHub, Stripe, a greenhouse sensor) can't |
| +encrypt to a thread key — but their events belong in the product. This design |
| +lets an admin hand an outside service a URL such that every HTTP request it |
| +sends becomes an **encrypted, typed event** (`webhooks.<ns>`) in **one chosen |
| +thread** — with the server never seeing plaintext content and the receiving |
| +process never able to read anything back. |
| + |
| +## The shape: an inverted bot sidecar |
| + |
| +The bot sidecar (`lchat bot run`, docs/bots.md) is a **pure outbound client**: |
| +it dials out, holds keys, and must never listen. The hooks sidecar |
| +(`lchat hooks run`, `internal/hooks/`) is its inverse — it exists to accept |
| +inbound connections, and is **strictly write-only** against the instance: |
| + |
| +- it never opens a websocket and never calls `GET /api/messages` — nothing in |
| + the process can read a conversation; |
| +- it never holds the instance key, so it can't decrypt profile blobs, member |
| + apps, charters, or anything else instance-key-sealed; |
| +- per webhook it holds exactly that webhook's private key and device token, |
| + which unwrap exactly **one** thread's key (to encrypt, and only encrypt). |
| + |
| +The two invariants stay unconditional by staying in separate processes: the |
| +bot sidecar remains "no inbound ports, relocatable anywhere"; the hooks |
| +sidecar is inbound-by-purpose and lives behind the router. Compromising the |
| +hooks process yields the threads its webhooks feed — never the rest of the |
| +space, never a bot's keys, never an Anthropic key (it has none). |
| + |
| +## Reaching it: `in.<host>`, a plain registry route |
| + |
| +Each instance's ingest surface is a sibling host — `in.family.11ch.at` for |
| +`family.11ch.at` — which is just a router registry row |
| +(`kind: "hooks"`, socket `/run/lchat/<host>-hooks.sock`). No router code |
| +special-cases it: the router proxies any registered host to its socket. |
| + |
| +TLS/DNS: the `*.11ch.at` **wildcard certificate** covers only one label, so |
| +`in.<sub>.11ch.at` takes the router's **per-host autocert** path instead — the |
| +cert is minted via HTTP-01 on first contact (HostPolicy allows it because the |
| +host is in the registry). Wildcard **DNS** does resolve two labels down when |
| +no closer record exists, so 11ch.at spaces need no zone change. (This is why |
| +the earlier `/hooks/*` path-split design — PR #617 — is superseded: the |
| +subdomain shape needs no router changes at all. The one real cost: a |
| +**custom-domain** space (`chat.paulca.com`) needs its owner to add an |
| +`in.chat.paulca.com` DNS record pointing at the same box.) |
| + |
| +The URL an outside service gets is `https://in.<host>/h/<secret>` — the |
| +secret is the capability, exactly the GitHub/Slack convention. The sidecar |
| +alone knows which secrets are live; the router forwards blindly and unknown |
| +secrets 404 **at the sidecar**, so the server can't even enumerate webhooks. |
| + |
| +## One webhook = one agent identity = one thread |
| + |
| +A webhook is an ordinary **agent member** (roster-visible, labelled with the |
| +webhook's name, badged `webhook`), created by the admin's browser: |
| + |
| +1. The admin opens the **Webhooks** app (admin-only, `/webhooks` or the Apps |
| + picker) and names a webhook + its **event namespace** (`github` → |
| + events of type `webhooks.github`) + its **target thread** (a group they're |
| + in, or the space thread). |
| +2. Their browser mints the webhook's **keypair**, wraps the target thread's |
| + key to it, and calls `POST /api/hooks` (admin-gated) — the server creates |
| + the agent user, mints its device token (returned once), and writes the |
| + membership row carrying the wrapped key. For the **space thread** this is |
| + the explicit-membership opt-in `threadMember` was built to honour: agents |
| + are fenced out of implicit membership and `open_key`, and a webhook gets |
| + the space key only because an admin deliberately wrapped it to that one |
| + identity. |
| +3. The browser generates the URL **secret**, then seals |
| + `{id, user_id, name, ns, thread_id, secret, token, priv}` to the **hooks |
| + service identity's pubkey** (ECIES `sealForPub`) → `def_blob`, and encrypts |
| + `{name, ns, thread_id, secret}` under the **instance key** → `meta_blob` |
| + (the app's list view). `PUT /api/hooks/{id}/blobs` stores both. The server |
| + holds only ciphertext: no key, no token plaintext (tokens are stored |
| + hashed), no secret, no target service. |
| + |
| +The sidecar polls `GET /api/hooks/defs` (~10s, sidecar-marked), opens each |
| +definition with its service-identity private key, materializes its route |
| +table, and acks new webhooks via `POST /api/hooks/{id}/deployed` — a plaintext |
| +timestamp, the only thing the server learns, which the app shows as |
| +"Deploying… → Active". Because the client minted the secret, the URL is |
| +displayed **instantly** at creation; the ack just confirms the route is |
| +serving. (#610's discussion had the sidecar minting the URL and feeding it |
| +back; client-minting is the deliberate simplification — a sidecar-minted URL |
| +would have to travel back encrypted, which needs the sidecar to hold the |
| +instance key, the very thing this design keeps away from it.) |
| + |
| +**Removal** (`DELETE /api/hooks/{id}`) deletes the row and sweeps the whole |
| +identity (`removeUser`: token, memberships, roster row); the sidecar's next |
| +poll drops the route and the URL dies. |
| + |
| +The **service identity** is provisioned once per instance by |
| +`lchat add-hooks <host>` (box-side, mirrors `add-bot`): it mints an agent |
| +invite flagged `hooks_service` — recorded in config as `hooks_service_user`, |
| +served to members at `GET /api/hooks/service` as the seal target — writes |
| +`/data/<host>-hooks.env`, registers the `in.<host>` route, and enables |
| +`lchat-hooks@<host>`. The sidecar self-provisions from the invite on first |
| +boot (`bot.EnsureSession`, the #806 zero-touch path). It's exempt from the |
| +bots gate: webhooks are integrations with their own admin-only gate, not AI. |
| + |
| +## The event: dumb ingest, smart render |
| + |
| +On each inbound request the sidecar builds (pinned by |
| +`test/event-vectors/webhooks.json`): |
| + |
| +```json |
| +{ |
| + "type": "webhooks.<ns>", |
| + "name": "<webhook name>", "uid": "<webhook user id>", "ts": 1752710400000, |
| + "hook": { "id": "wh-…", "ns": "<ns>" }, |
| + "req": { |
| + "method": "POST", "path": "/<subpath after the secret>", |
| + "query": { "site": ["7"] }, |
| + "headers": { "Content-Type": "…", "X-Github-Event": "…" }, |
| + "body": "<raw body>", "body_b64": false |
| + } |
| +} |
| +``` |
| + |
| +encrypts it to the thread key, and `POST /api/messages` as the webhook — the |
| +Announcer's outbound-only walk. The sidecar **never transforms**: the raw |
| +request rides inside the event forever, so rendering can improve later without |
| +re-ingesting. |
| + |
| +- **No default rendering.** Webhook events are consumed by *apps* — a CI |
| + dashboard app claims exactly `webhooks.github` in its `events` map and folds |
| + from there. Until an app claims a type, every client silently ignores it: |
| + web via `foldEvent`/`isAppType` (no owner → dropped), native via |
| + `MessagePayload.isRenderableChat` (typed non-chat → not rendered, not |
| + notified), the service worker via its typed-events-stay-silent rule. Old |
| + clients degrade to exactly this. The management app itself claims **no** |
| + types. |
| +- **Headers are allowlisted** (`Content-Type`, `User-Agent`, provider |
| + event/signature headers). Full headers routinely carry sender credentials |
| + (`Authorization`, cookies) that must never enter the log — even encrypted, |
| + every thread member could read them. `req.path` is the sub-path *after* the |
| + secret segment, so the capability never rides inside an event. |
| +- **Notifications:** space-thread deliveries never push at all (push.go skips |
| + `kind='space'`); group deliveries push ciphertext but render nothing (typed, |
| + unclaimed) — so a webhook can't buzz a phone until a consumer app decides |
| + events are worth words. |
| + |
| +## Ingest hygiene |
| + |
| +- Unknown secret → **404**; nothing distinguishes "never existed" from |
| + "removed". |
| +- Body over ~40 KB → **413**, never truncated (the encrypted event must fit |
| + the server's 64 KB message-blob cap). |
| +- Per-webhook token bucket (60 burst, 1/s sustained) → **429**; the server's |
| + content-blind message brake backstops it independently. |
| +- Non-UTF-8 bodies ride base64url with `body_b64: true`. |
| +- Delivery failure upstream → **502**, so senders with retry logic retry. |
| +- Hibernation-aware: the recurring definition poll is sidecar-marked (never |
| + holds a sleeping instance awake; the last-known route table keeps serving), |
| + while an actual delivery is unmarked — a webhook arrival is real external |
| + activity and **wakes** the instance to land its event. |
| + |
| +## Dev story |
| + |
| +No router locally: `lchat hooks run --addr :8090 --base http://localhost:1747` |
| +listens on TCP directly (`LCHAT_INVITE` from a hand-minted agent invite). The |
| +webhooks app displays `in.<page-host>` URLs regardless; point the sender at |
| +the dev listener. |
| + |
| +## Later (deliberately not in v1) |
| + |
| +- **Member-mintable webhooks** with quotas (creation is admin-only today). |
| +- **Sender signature enforcement** (e.g. verify `X-Hub-Signature-256` at the |
| + sidecar with a per-webhook shared secret; reject unsigned). |
| +- **Consumer apps** — the point of the exercise: e.g. a GitHub Actions |
| + dashboard claiming `webhooks.github`, parsing `workflow_run` payloads into |
| + a live runs board. |
| +- **Multi-thread webhooks / re-targeting** — v1 binds one webhook to one |
| + thread at creation; re-point by remove + recreate. |
diff --git a/hooks.go b/hooks.go| new file mode 100644 |
| index 0000000..ffae7ce |
| --- /dev/null |
| +++ b/hooks.go |
| @@ -0,0 +1,310 @@ |
| +package main |
| + |
| +// Incoming webhooks (issue #610, docs/webhooks.md): the server half of the |
| +// hooks sidecar (`lchat hooks run`) — the INVERSE of the bot sidecar. Where a |
| +// bot is a pure-outbound member that must never listen, a webhook is a |
| +// pure-INBOUND relay: external services POST plaintext to `in.<host>`, the |
| +// sidecar encrypts each request into a typed event (`webhooks.<ns>`) and posts |
| +// it into the one thread that webhook belongs to. The sidecar can never read |
| +// anything back — it never opens a websocket, never fetches messages, and |
| +// never holds the instance key. |
| +// |
| +// The server's role is the member-bots one: store OPAQUE blobs and mint the |
| +// identities' plumbing. |
| +// |
| +// - Each webhook is an ordinary agent member (roster-visible, labelled with |
| +// the webhook's name) whose keypair is minted by the CREATING ADMIN'S |
| +// BROWSER. Its private key + device token + URL secret travel inside |
| +// def_blob, sealed (ECIES) to the hooks sidecar's service identity — the |
| +// server stores ciphertext it cannot open and never sees a usable key. |
| +// - meta_blob (name, namespace, thread, secret → the displayed URL) is |
| +// instance-key-encrypted for the admin app's list view. |
| +// - Thread membership is the routing: the creating admin wraps the thread |
| +// key to the webhook's pubkey, and the webhook may post only there. The |
| +// space thread works via an EXPLICIT membership row — the deliberate |
| +// agent opt-in threadMember() already anticipates. |
| +// |
| +// Management is admin-only (requireAdmin: the break-glass admin token or an |
| +// admin member's own device token). The sidecar authenticates with its agent |
| +// device token; the defs it polls are protected by crypto, not by the gate — |
| +// any member could fetch the ciphertext, only the service identity can open it. |
| + |
| +import ( |
| + "encoding/json" |
| + "net/http" |
| + "strings" |
| +) |
| + |
| +const ( |
| + // maxWebhookBlobBytes caps each encrypted blob: a definition or meta blob |
| + // is well under 2 KB of JSON; 16 K is lavish headroom. |
| + maxWebhookBlobBytes = 16 * 1024 |
| + // maxWebhooks bounds the registry — every webhook is a roster identity and |
| + // a standing inbound surface, so the cap is an abuse dial like member bots'. |
| + maxWebhooks = 50 |
| + // hooksServiceUserKey is the config row naming the sidecar's service |
| + // identity (set when its agent invite is minted with hooks_service, see |
| + // createKeyInvite). Its pubkey is what creation seals definitions to. |
| + hooksServiceUserKey = "hooks_service_user" |
| +) |
| + |
| +// getHooksService tells a member whether this space has a hooks sidecar and, if |
| +// so, which roster identity it is — the seal target for definitions. {} when |
| +// none is provisioned (the webhooks app shows its not-set-up empty state). |
| +// Member-token-gated like the roster: the pubkey is roster-public anyway. |
| +func (s *Server) getHooksService(w http.ResponseWriter, r *http.Request) { |
| + if !s.tokenOK(r.URL.Query().Get("token")) { |
| + httpError(w, http.StatusForbidden, "invalid token") |
| + return |
| + } |
| + uid, ok, err := s.db.getConfig(hooksServiceUserKey) |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + if !ok || uid == "" || !s.db.userExists(uid) { |
| + writeJSON(w, http.StatusOK, map[string]any{}) |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"user_id": uid, "public_key": s.db.userPublicKey(uid)}) |
| +} |
| + |
| +// createWebhookHandler mints a webhook: the agent user row (plaintext label, |
| +// like bot labels — the roster discloses the webhook by name), its device |
| +// token (returned ONCE, for the creator to seal into the definition), and its |
| +// thread membership carrying the wrapped thread key. The blobs land in a |
| +// second step (putWebhookBlobsHandler) because the definition can't be sealed |
| +// until the token exists. |
| +func (s *Server) createWebhookHandler(w http.ResponseWriter, r *http.Request) { |
| + if !s.requireAdmin(w, r) { |
| + return |
| + } |
| + var body struct { |
| + Name string `json:"name"` |
| + PublicKey string `json:"public_key"` |
| + ThreadID string `json:"thread_id"` |
| + Wrapped string `json:"wrapped"` |
| + } |
| + if json.NewDecoder(http.MaxBytesReader(w, r.Body, 16*1024)).Decode(&body) != nil { |
| + httpError(w, http.StatusBadRequest, "bad request") |
| + return |
| + } |
| + name := strings.TrimSpace(body.Name) |
| + if name == "" || len(name) > 64 { |
| + httpError(w, http.StatusBadRequest, "name must be 1-64 characters") |
| + return |
| + } |
| + if body.PublicKey == "" || len(body.PublicKey) > 200 || body.Wrapped == "" || len(body.Wrapped) > 4096 { |
| + httpError(w, http.StatusBadRequest, "bad key material") |
| + return |
| + } |
| + switch s.db.threadKind(body.ThreadID) { |
| + case "group", "space": |
| + default: |
| + // DM/self threads keep their one-thread-per-pair invariants; a webhook |
| + // feeds a group or the space thread. |
| + httpError(w, http.StatusBadRequest, "webhooks post into a group or the space thread") |
| + return |
| + } |
| + // When the caller is an admin MEMBER (not the break-glass console token), |
| + // they must be in the target thread — that's whose browser wrapped the |
| + // thread key, so it's also the only honest case. |
| + if uid, ok := s.db.deviceTokenUser(strings.TrimSpace(strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer "))); ok { |
| + if !s.db.threadMember(body.ThreadID, uid) { |
| + httpError(w, http.StatusForbidden, "not a member of that thread") |
| + return |
| + } |
| + } |
| + if n, err := s.db.countWebhooks(); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } else if n >= maxWebhooks { |
| + httpError(w, http.StatusForbidden, "webhook limit reached") |
| + return |
| + } |
| + // The webhook's identity: an agent user labelled with the webhook's name. |
| + // Deliberately NOT gated on botsEnabled — webhooks are integrations with |
| + // their own admin-only gate, not AI bots. |
| + userID, err := s.db.createUser(name) |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + // Failures past this point best-effort sweep the half-made identity so a |
| + // retry doesn't strand roster rows (removeUser clears tokens + memberships). |
| + fail := func() { |
| + _ = s.db.removeUser(userID) |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + } |
| + if err := s.db.setUserAgent(userID); err != nil { |
| + fail() |
| + return |
| + } |
| + // The roster badge: a distinct kind so clients can render webhooks as what |
| + // they are (declareBotKind stays bots-only; this is server-assigned). |
| + if err := s.db.setBotKind(userID, "webhook"); err != nil { |
| + fail() |
| + return |
| + } |
| + // The creator's browser minted the keypair, so the pubkey registers here |
| + // (not via POST /api/pubkey — the webhook has no session of its own yet). |
| + // No bind-MAC: webhook identities aren't invite-bootstrapped, so there is |
| + // no invite secret to bind key transparency to; the KT join event still |
| + // journals the key. |
| + if err := s.db.setUserPublicKey(userID, body.PublicKey, ""); err != nil { |
| + fail() |
| + return |
| + } |
| + tok, err := s.db.createDeviceToken(userID, "") |
| + if err != nil { |
| + fail() |
| + return |
| + } |
| + // Membership = routing. For the space thread this is the explicit |
| + // membership row that opts an agent in (threadMember checks rows first); |
| + // for groups it's the ordinary add-member write. |
| + if err := s.db.addThreadMember(body.ThreadID, userID, body.Wrapped); err != nil { |
| + fail() |
| + return |
| + } |
| + id := "wh-" + userID[:16] |
| + if err := s.db.createWebhook(id, userID); err != nil { |
| + fail() |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"id": id, "user_id": userID, "token": tok}) |
| +} |
| + |
| +// putWebhookBlobsHandler stores the two encrypted blobs — the create flow's |
| +// second step, once the creator has sealed the definition around the token. |
| +func (s *Server) putWebhookBlobsHandler(w http.ResponseWriter, r *http.Request) { |
| + if !s.requireAdmin(w, r) { |
| + return |
| + } |
| + var body struct { |
| + DefBlob string `json:"def_blob"` |
| + MetaBlob string `json:"meta_blob"` |
| + } |
| + if json.NewDecoder(http.MaxBytesReader(w, r.Body, 2*maxWebhookBlobBytes+4096)).Decode(&body) != nil { |
| + httpError(w, http.StatusBadRequest, "bad request") |
| + return |
| + } |
| + if body.DefBlob == "" || len(body.DefBlob) > maxWebhookBlobBytes || |
| + body.MetaBlob == "" || len(body.MetaBlob) > maxWebhookBlobBytes { |
| + httpError(w, http.StatusBadRequest, "blob missing or too large") |
| + return |
| + } |
| + ok, err := s.db.setWebhookBlobs(r.PathValue("id"), body.DefBlob, body.MetaBlob) |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + if !ok { |
| + httpError(w, http.StatusNotFound, "no such webhook") |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"ok": true}) |
| +} |
| + |
| +// listWebhooksHandler is the admin app's list view: meta blobs (which only |
| +// members can decrypt) plus the plaintext deploy state. |
| +func (s *Server) listWebhooksHandler(w http.ResponseWriter, r *http.Request) { |
| + if !s.requireAdmin(w, r) { |
| + return |
| + } |
| + hooks, err := s.db.listWebhooks() |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"hooks": hooks}) |
| +} |
| + |
| +// hooksServiceCaller reports whether tok belongs to the RECORDED hooks service |
| +// identity — the narrowest honest gate for the sidecar-facing endpoints. (The |
| +// real protection on definitions is still the seal: only the service |
| +// identity's private key opens one.) |
| +func (s *Server) hooksServiceCaller(tok string) bool { |
| + uid, ok := s.db.deviceTokenUser(tok) |
| + if !ok || !s.db.isAgent(uid) { |
| + return false |
| + } |
| + svc, found, err := s.db.getConfig(hooksServiceUserKey) |
| + return err == nil && found && svc == uid |
| +} |
| + |
| +// listWebhookDefsHandler is the sidecar's poll: definition blobs only. |
| +func (s *Server) listWebhookDefsHandler(w http.ResponseWriter, r *http.Request) { |
| + if !s.hooksServiceCaller(r.URL.Query().Get("token")) { |
| + httpError(w, http.StatusForbidden, "hooks service token required") |
| + return |
| + } |
| + hooks, err := s.db.listWebhookDefs() |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"hooks": hooks}) |
| +} |
| + |
| +// webhookDeployedHandler records the sidecar's "route is live" ack — plaintext |
| +// by design (a timestamp, no content), the signal the admin app shows as |
| +// Deploying… → Active. |
| +func (s *Server) webhookDeployedHandler(w http.ResponseWriter, r *http.Request) { |
| + var body struct { |
| + Token string `json:"token"` |
| + } |
| + if json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&body) != nil { |
| + httpError(w, http.StatusBadRequest, "bad request") |
| + return |
| + } |
| + if !s.hooksServiceCaller(body.Token) { |
| + httpError(w, http.StatusForbidden, "hooks service token required") |
| + return |
| + } |
| + id := r.PathValue("id") |
| + if _, exists, err := s.db.getWebhook(id); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } else if !exists { |
| + httpError(w, http.StatusNotFound, "no such webhook") |
| + return |
| + } |
| + if err := s.db.setWebhookDeployed(id); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"ok": true}) |
| +} |
| + |
| +// deleteWebhookHandler retires a webhook: the row goes, and so does its whole |
| +// identity — removeUser sweeps the device token, memberships, and roster row, |
| +// so the sidecar's next poll drops the route and the URL dies with it. |
| +func (s *Server) deleteWebhookHandler(w http.ResponseWriter, r *http.Request) { |
| + if !s.requireAdmin(w, r) { |
| + return |
| + } |
| + id := r.PathValue("id") |
| + wh, exists, err := s.db.getWebhook(id) |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + if !exists { |
| + httpError(w, http.StatusNotFound, "no such webhook") |
| + return |
| + } |
| + if _, err := s.db.deleteWebhook(id); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + // Sweep the identity only if it really is an agent — a belt-and-braces |
| + // guard so this cascade can never take a person's account with it. |
| + if s.db.isAgent(wh.UserID) { |
| + if err := s.db.removeUser(wh.UserID); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + } |
| + writeJSON(w, http.StatusOK, map[string]any{"ok": true}) |
| +} |
diff --git a/hooks_test.go b/hooks_test.go| new file mode 100644 |
| index 0000000..eb5f201 |
| --- /dev/null |
| +++ b/hooks_test.go |
| @@ -0,0 +1,425 @@ |
| +package main |
| + |
| +// End-to-end tests for incoming webhooks (hooks.go + internal/hooks): the |
| +// admin's create flow, the sidecar's poll → deploy → ingest → encrypt → post |
| +// walk against the real router, and the gates around it all. |
| + |
| +import ( |
| + "bytes" |
| + "encoding/json" |
| + "fmt" |
| + "net/http" |
| + "net/http/httptest" |
| + "strings" |
| + "testing" |
| + |
| + "lchat/internal/bot" |
| + "lchat/internal/e2ee" |
| + "lchat/internal/hooks" |
| +) |
| + |
| +// provisionHooksService mints the service identity's invite (hooks_service, as |
| +// add-hooks does) and claims it, returning the sidecar's session. |
| +func provisionHooksService(t *testing.T, tsURL string) *bot.Session { |
| + t.Helper() |
| + secret, err := e2ee.RandomSecret() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + inv, err := e2ee.DeriveInvite(secret) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + body, _ := json.Marshal(map[string]any{ |
| + "label": "Webhooks", |
| + "id": inv.ID, |
| + "claim_hash": inv.ClaimHash, |
| + "agent": true, |
| + "hooks_service": true, |
| + }) |
| + req, _ := http.NewRequest("POST", tsURL+"/api/admin/key-invites", bytes.NewReader(body)) |
| + req.Header.Set("Authorization", "Bearer admintok") |
| + req.Header.Set("Content-Type", "application/json") |
| + resp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer resp.Body.Close() |
| + var out struct { |
| + OK bool `json:"ok"` |
| + } |
| + if json.NewDecoder(resp.Body).Decode(&out) != nil || !out.OK { |
| + t.Fatalf("mint hooks-service invite: status %d", resp.StatusCode) |
| + } |
| + sess, err := bot.Provision(tsURL, secret, "Webhooks") |
| + if err != nil { |
| + t.Fatalf("provision service identity: %v", err) |
| + } |
| + return sess |
| +} |
| + |
| +// mkAdminMember creates a joined admin member with a device token. |
| +func mkAdminMember(t *testing.T, s *Server, label string) (token, uid string) { |
| + t.Helper() |
| + tok := mintDeviceToken(t, s.db, label) |
| + uid, ok := s.db.deviceTokenUser(tok) |
| + if !ok { |
| + t.Fatal("no uid for minted token") |
| + } |
| + if err := s.db.setUserRole(uid, "admin"); err != nil { |
| + t.Fatal(err) |
| + } |
| + return tok, uid |
| +} |
| + |
| +// createWebhookAsAdmin walks the app's create flow: keypair, wrap the thread |
| +// key, POST /api/hooks, seal the definition to the service identity, PUT blobs. |
| +func createWebhookAsAdmin(t *testing.T, tsURL, adminBearer, threadID, threadKeyB64, servicePub, name, ns string) (id, secret string) { |
| + t.Helper() |
| + kp, err := e2ee.GenerateUserKeypair() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + wrapped, err := e2ee.WrapThreadKey(threadKeyB64, kp.PublicKey) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + body, _ := json.Marshal(map[string]string{ |
| + "name": name, "public_key": kp.PublicKey, "thread_id": threadID, "wrapped": wrapped, |
| + }) |
| + req, _ := http.NewRequest("POST", tsURL+"/api/hooks", bytes.NewReader(body)) |
| + req.Header.Set("Authorization", "Bearer "+adminBearer) |
| + resp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer resp.Body.Close() |
| + var created struct { |
| + ID string `json:"id"` |
| + UserID string `json:"user_id"` |
| + Token string `json:"token"` |
| + } |
| + if resp.StatusCode != http.StatusOK { |
| + t.Fatalf("POST /api/hooks: status %d", resp.StatusCode) |
| + } |
| + if err := json.NewDecoder(resp.Body).Decode(&created); err != nil || created.ID == "" || created.Token == "" { |
| + t.Fatalf("create response: %+v err=%v", created, err) |
| + } |
| + |
| + secret, err = e2ee.RandomSecret() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + def, err := e2ee.SealForPub(servicePub, hooks.Definition{ |
| + V: 1, ID: created.ID, UserID: created.UserID, Name: name, NS: ns, |
| + ThreadID: threadID, Secret: secret, Token: created.Token, Priv: kp.PrivateKey, |
| + }) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + blobs, _ := json.Marshal(map[string]string{"def_blob": def, "meta_blob": "opaque-meta"}) |
| + req, _ = http.NewRequest("PUT", tsURL+"/api/hooks/"+created.ID+"/blobs", bytes.NewReader(blobs)) |
| + req.Header.Set("Authorization", "Bearer "+adminBearer) |
| + resp2, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer resp2.Body.Close() |
| + if resp2.StatusCode != http.StatusOK { |
| + t.Fatalf("PUT blobs: status %d", resp2.StatusCode) |
| + } |
| + return created.ID, secret |
| +} |
| + |
| +// lastEventInThread fetches and decrypts the newest message in a thread. |
| +func lastEventInThread(t *testing.T, tsURL, memberTok, threadID, threadKeyB64 string) map[string]any { |
| + t.Helper() |
| + resp, err := http.Get(tsURL + "/api/messages?token=" + memberTok + "&thread=" + threadID) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer resp.Body.Close() |
| + var out struct { |
| + Messages []struct { |
| + Blob string `json:"blob"` |
| + } `json:"messages"` |
| + } |
| + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(out.Messages) == 0 { |
| + t.Fatal("no messages in thread") |
| + } |
| + key, err := e2ee.ImportKey(threadKeyB64) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var payload map[string]any |
| + if err := e2ee.DecryptJSON(key, out.Messages[len(out.Messages)-1].Blob, &payload); err != nil { |
| + t.Fatalf("decrypt event: %v", err) |
| + } |
| + return payload |
| +} |
| + |
| +func TestWebhookEndToEnd(t *testing.T) { |
| + ts, s := newTestServer(t) |
| + sess := provisionHooksService(t, ts.URL) |
| + |
| + // The service identity is discoverable (the seal target for creation). |
| + resp, err := http.Get(ts.URL + "/api/hooks/service?token=" + sess.Token) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var svc struct { |
| + UserID string `json:"user_id"` |
| + PublicKey string `json:"public_key"` |
| + } |
| + json.NewDecoder(resp.Body).Decode(&svc) |
| + resp.Body.Close() |
| + if svc.UserID == "" || svc.PublicKey != sess.UserPub { |
| + t.Fatalf("hooks service: %+v (want the provisioned identity)", svc) |
| + } |
| + |
| + // An admin member with a group thread (the webhook's target). |
| + adminTok, adminUID := mkAdminMember(t, s, "Paula") |
| + threadKey, _ := e2ee.GenerateKey() |
| + threadKeyB64 := e2ee.B64uEncode(threadKey) |
| + body := `{"token":"` + adminTok + `","kind":"group","members":[{"user_id":"` + adminUID + `","wrapped":"wa"}]}` |
| + tresp, err := http.Post(ts.URL+"/api/threads", "application/json", strings.NewReader(body)) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var tout struct { |
| + ThreadID string `json:"thread_id"` |
| + } |
| + json.NewDecoder(tresp.Body).Decode(&tout) |
| + tresp.Body.Close() |
| + if tout.ThreadID == "" { |
| + t.Fatal("no thread id") |
| + } |
| + |
| + id, secret := createWebhookAsAdmin(t, ts.URL, adminTok, tout.ThreadID, threadKeyB64, svc.PublicKey, "CI runs", "github") |
| + |
| + // The webhook identity appears in the thread and on the roster as an agent. |
| + wh, exists, _ := s.db.getWebhook(id) |
| + if !exists || !s.db.isAgent(wh.UserID) || !s.db.threadMember(tout.ThreadID, wh.UserID) { |
| + t.Fatalf("webhook identity not fully materialized: exists=%v", exists) |
| + } |
| + |
| + // Sidecar: poll → route live + deployed acked. |
| + sc := hooks.New(ts.URL, sess, t.Logf) |
| + if err := sc.PollOnce(); err != nil { |
| + t.Fatalf("poll: %v", err) |
| + } |
| + if sc.Routes() != 1 { |
| + t.Fatalf("routes = %d, want 1", sc.Routes()) |
| + } |
| + if wh, _, _ := s.db.getWebhook(id); wh.DeployedAt == 0 { |
| + t.Error("deployed_at not acked") |
| + } |
| + |
| + ingest := httptest.NewServer(sc.Handler()) |
| + defer ingest.Close() |
| + |
| + // Deliver: an allowlisted header rides in, Authorization must not. |
| + req, _ := http.NewRequest("POST", ingest.URL+"/h/"+secret, strings.NewReader(`{"action":"completed"}`)) |
| + req.Header.Set("Content-Type", "application/json") |
| + req.Header.Set("X-GitHub-Event", "workflow_run") |
| + req.Header.Set("Authorization", "Bearer sender-secret") |
| + dresp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + dresp.Body.Close() |
| + if dresp.StatusCode != http.StatusOK { |
| + t.Fatalf("delivery: status %d", dresp.StatusCode) |
| + } |
| + |
| + event := lastEventInThread(t, ts.URL, adminTok, tout.ThreadID, threadKeyB64) |
| + if event["type"] != "webhooks.github" { |
| + t.Errorf("type = %v, want webhooks.github", event["type"]) |
| + } |
| + if event["name"] != "CI runs" || event["uid"] != wh.UserID { |
| + t.Errorf("attribution: name=%v uid=%v", event["name"], event["uid"]) |
| + } |
| + reqPart, _ := event["req"].(map[string]any) |
| + if reqPart == nil { |
| + t.Fatalf("no req in event: %v", event) |
| + } |
| + if reqPart["method"] != "POST" || reqPart["body"] != `{"action":"completed"}` { |
| + t.Errorf("req = %v", reqPart) |
| + } |
| + headers, _ := reqPart["headers"].(map[string]any) |
| + if headers["X-Github-Event"] != "workflow_run" { |
| + t.Errorf("allowlisted header missing: %v", headers) |
| + } |
| + for k := range headers { |
| + if strings.EqualFold(k, "Authorization") { |
| + t.Error("Authorization header captured into the event") |
| + } |
| + } |
| + |
| + // Ingest hygiene: unknown secret 404s, oversize 413s. |
| + if r, _ := http.Post(ingest.URL+"/h/nope", "text/plain", strings.NewReader("x")); r.StatusCode != http.StatusNotFound { |
| + t.Errorf("unknown secret: status %d, want 404", r.StatusCode) |
| + } |
| + if r, _ := http.Post(ingest.URL+"/h/"+secret, "text/plain", strings.NewReader(strings.Repeat("a", 41*1024))); r.StatusCode != http.StatusRequestEntityTooLarge { |
| + t.Errorf("oversize body: status %d, want 413", r.StatusCode) |
| + } |
| + |
| + // Delete retires the whole identity; the next poll drops the route and the |
| + // URL dies. |
| + dreq, _ := http.NewRequest("DELETE", ts.URL+"/api/hooks/"+id, nil) |
| + dreq.Header.Set("Authorization", "Bearer "+adminTok) |
| + delResp, err := http.DefaultClient.Do(dreq) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + delResp.Body.Close() |
| + if delResp.StatusCode != http.StatusOK { |
| + t.Fatalf("delete: status %d", delResp.StatusCode) |
| + } |
| + if s.db.userExists(wh.UserID) { |
| + t.Error("webhook user survived deletion") |
| + } |
| + if err := sc.PollOnce(); err != nil { |
| + t.Fatalf("poll after delete: %v", err) |
| + } |
| + if sc.Routes() != 0 { |
| + t.Errorf("routes after delete = %d, want 0", sc.Routes()) |
| + } |
| + if r, _ := http.Post(ingest.URL+"/h/"+secret, "text/plain", strings.NewReader("x")); r.StatusCode != http.StatusNotFound { |
| + t.Errorf("deleted webhook still ingesting: status %d", r.StatusCode) |
| + } |
| +} |
| + |
| +func TestWebhookSpaceThread(t *testing.T) { |
| + ts, s := newTestServer(t) |
| + sess := provisionHooksService(t, ts.URL) |
| + adminTok, _ := mkAdminMember(t, s, "Paula") |
| + |
| + // The space thread exists with an open_key only members can unwrap; the |
| + // webhook gets the SAME thread key wrapped to its own pubkey instead — the |
| + // explicit agent opt-in. |
| + spaceID, _, created, err := s.db.createSpaceThread("opaque-open-key") |
| + if err != nil || !created { |
| + t.Fatalf("createSpaceThread: created=%v err=%v", created, err) |
| + } |
| + threadKey, _ := e2ee.GenerateKey() |
| + threadKeyB64 := e2ee.B64uEncode(threadKey) |
| + |
| + id, secret := createWebhookAsAdmin(t, ts.URL, adminTok, spaceID, threadKeyB64, sess.UserPub, "Deploy feed", "github") |
| + |
| + sc := hooks.New(ts.URL, sess, t.Logf) |
| + if err := sc.PollOnce(); err != nil { |
| + t.Fatalf("poll: %v", err) |
| + } |
| + ingest := httptest.NewServer(sc.Handler()) |
| + defer ingest.Close() |
| + |
| + if r, _ := http.Post(ingest.URL+"/h/"+secret, "application/json", strings.NewReader(`{"run":1}`)); r.StatusCode != http.StatusOK { |
| + t.Fatalf("space delivery: status %d", r.StatusCode) |
| + } |
| + event := lastEventInThread(t, ts.URL, adminTok, spaceID, threadKeyB64) |
| + if event["type"] != "webhooks.github" { |
| + t.Errorf("type = %v", event["type"]) |
| + } |
| + _ = id |
| +} |
| + |
| +func TestWebhookGates(t *testing.T) { |
| + ts, s := newTestServer(t) |
| + |
| + // A plain member can neither create nor list webhooks. |
| + memberTok := mintDeviceToken(t, s.db, "Mallory") |
| + body, _ := json.Marshal(map[string]string{"name": "x", "public_key": "p", "thread_id": "t", "wrapped": "w"}) |
| + req, _ := http.NewRequest("POST", ts.URL+"/api/hooks", bytes.NewReader(body)) |
| + req.Header.Set("Authorization", "Bearer "+memberTok) |
| + resp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + resp.Body.Close() |
| + if resp.StatusCode != http.StatusUnauthorized { |
| + t.Errorf("member create: status %d, want 401", resp.StatusCode) |
| + } |
| + |
| + // The defs poll needs an agent token — a person's token is refused. |
| + r, err := http.Get(ts.URL + "/api/hooks/defs?token=" + memberTok) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusForbidden { |
| + t.Errorf("member defs poll: status %d, want 403", r.StatusCode) |
| + } |
| + |
| + // Creating into a DM is refused even for an admin (webhooks feed groups and |
| + // the space thread only). |
| + adminTok, adminUID := mkAdminMember(t, s, "Paula") |
| + dmBody := `{"token":"` + adminTok + `","kind":"dm","members":[{"user_id":"` + adminUID + `","wrapped":"wa"},{"user_id":"` + mustUID(t, s, memberTok) + `","wrapped":"wb"}]}` |
| + tresp, err := http.Post(ts.URL+"/api/threads", "application/json", strings.NewReader(dmBody)) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var tout struct { |
| + ThreadID string `json:"thread_id"` |
| + } |
| + json.NewDecoder(tresp.Body).Decode(&tout) |
| + tresp.Body.Close() |
| + body, _ = json.Marshal(map[string]string{"name": "x", "public_key": "p", "thread_id": tout.ThreadID, "wrapped": "w"}) |
| + req, _ = http.NewRequest("POST", ts.URL+"/api/hooks", bytes.NewReader(body)) |
| + req.Header.Set("Authorization", "Bearer "+adminTok) |
| + resp, err = http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + resp.Body.Close() |
| + if resp.StatusCode != http.StatusBadRequest { |
| + t.Errorf("create into DM: status %d, want 400", resp.StatusCode) |
| + } |
| +} |
| + |
| +// TestWebhookRateLimit drains one webhook's bucket and expects 429 — the |
| +// sidecar-side dial; the server's message brake backstops it independently. |
| +func TestWebhookRateLimit(t *testing.T) { |
| + ts, s := newTestServer(t) |
| + sess := provisionHooksService(t, ts.URL) |
| + adminTok, adminUID := mkAdminMember(t, s, "Paula") |
| + threadKey, _ := e2ee.GenerateKey() |
| + threadKeyB64 := e2ee.B64uEncode(threadKey) |
| + body := `{"token":"` + adminTok + `","kind":"group","members":[{"user_id":"` + adminUID + `","wrapped":"wa"}]}` |
| + tresp, err := http.Post(ts.URL+"/api/threads", "application/json", strings.NewReader(body)) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var tout struct { |
| + ThreadID string `json:"thread_id"` |
| + } |
| + json.NewDecoder(tresp.Body).Decode(&tout) |
| + tresp.Body.Close() |
| + |
| + _, secret := createWebhookAsAdmin(t, ts.URL, adminTok, tout.ThreadID, threadKeyB64, sess.UserPub, "Chatty", "github") |
| + sc := hooks.New(ts.URL, sess, t.Logf) |
| + if err := sc.PollOnce(); err != nil { |
| + t.Fatal(err) |
| + } |
| + ingest := httptest.NewServer(sc.Handler()) |
| + defer ingest.Close() |
| + |
| + limited := false |
| + for i := 0; i < 100; i++ { |
| + r, err := http.Post(ingest.URL+"/h/"+secret, "text/plain", strings.NewReader(fmt.Sprintf("n=%d", i))) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + r.Body.Close() |
| + if r.StatusCode == http.StatusTooManyRequests { |
| + limited = true |
| + break |
| + } |
| + } |
| + if !limited { |
| + t.Error("100 rapid deliveries never rate-limited") |
| + } |
| +} |
diff --git a/hookscmd.go b/hookscmd.go| new file mode 100644 |
| index 0000000..4677973 |
| --- /dev/null |
| +++ b/hookscmd.go |
| @@ -0,0 +1,218 @@ |
| +// The `lchat hooks` subcommand family (issue #610, docs/webhooks.md): |
| +// |
| +// - `lchat hooks run` IS the webhook-ingest sidecar (internal/hooks) — run on |
| +// the router box behind an `in.<host>` registry route, or anywhere for dev |
| +// with -addr. |
| +// - `lchat add-hooks <host>` is the router-box control command that stands it |
| +// up for an instance: service-identity invite, EnvironmentFile, registry |
| +// route, systemd unit — the add-bot shape. |
| +package main |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "encoding/json" |
| + "flag" |
| + "fmt" |
| + "io" |
| + "log" |
| + "net/http" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "time" |
| + |
| + "lchat/internal/bot" |
| + "lchat/internal/e2ee" |
| + "lchat/internal/hooks" |
| +) |
| + |
| +// hooksEnvPath / hooksSessionDir: the per-instance convention, mirroring |
| +// botEnvPath — the EnvironmentFile add-hooks writes and the dir the sidecar |
| +// persists its service-identity session in. |
| +func hooksEnvPath(host string) string { return filepath.Join(instanceDataDir, host+"-hooks.env") } |
| +func hooksSessionDir(host string) string { return filepath.Join(instanceDataDir, host+"-hooks") } |
| + |
| +// hooksSocketPath is where the router reaches a host's hooks sidecar (the |
| +// in.<host> registry route's socket). |
| +func hooksSocketPath(host string) string { return "/run/lchat/" + host + "-hooks.sock" } |
| + |
| +func runHooksCmd(args []string) { |
| + if len(args) < 1 { |
| + fmt.Fprintln(os.Stderr, `usage: lchat hooks <command> |
| + |
| +commands: |
| + run run the webhook-ingest sidecar: accept inbound HTTP on the socket/addr, |
| + encrypt each request into a typed event, post it into its webhook's thread`) |
| + os.Exit(2) |
| + } |
| + switch args[0] { |
| + case "run": |
| + runHooksRun(args[1:]) |
| + default: |
| + die("hooks: unknown command %q", args[0]) |
| + } |
| +} |
| + |
| +func runHooksRun(args []string) { |
| + fs := flag.NewFlagSet("hooks run", flag.ExitOnError) |
| + base := fs.String("base", envOr("LCHAT_BASE", ""), "instance URL (env: LCHAT_BASE)") |
| + session := fs.String("session", envOr("LCHAT_SESSION", "hooks-session.json"), "service-identity session file (env: LCHAT_SESSION)") |
| + socket := fs.String("socket", envOr("LCHAT_SOCKET", ""), "unix socket to serve on instead of -addr (env: LCHAT_SOCKET); how the router fronts in.<host>") |
| + addr := fs.String("addr", envOr("ADDR", ""), "TCP listen address for local development (no router)") |
| + _ = fs.Parse(reorderFlagsFirst(fs, args)) |
| + |
| + if *base == "" { |
| + die("hooks run: set -base (or LCHAT_BASE) to your instance URL") |
| + } |
| + if *socket == "" && *addr == "" { |
| + die("hooks run: set -socket (production, behind the router) or -addr (dev)") |
| + } |
| + // Self-provision on first boot from the one-time invite add-hooks staged, |
| + // exactly like the bot sidecar (#806). The invite is env-only — a flag |
| + // would leak it into the process table. |
| + sess, err := bot.EnsureSession(strings.TrimRight(*base, "/"), *session, os.Getenv("LCHAT_INVITE"), "Webhooks", log.Printf) |
| + if err != nil { |
| + die("hooks run: %v", err) |
| + } |
| + if sess == nil { |
| + die("hooks run: no session at %s and no LCHAT_INVITE — run add-hooks (or mint an agent invite) first", *session) |
| + } |
| + |
| + sc := hooks.New(*base, sess, log.Printf) |
| + go sc.Run(context.Background(), hooks.DefaultPoll) |
| + |
| + ln, where, err := listen(*addr, *socket) |
| + if err != nil { |
| + die("hooks run: %v", err) |
| + } |
| + log.Printf("hooks run: ingesting on %s for %s (%d webhook(s) at last poll)", where, *base, sc.Routes()) |
| + srv := &http.Server{Handler: sc.Handler(), ReadHeaderTimeout: 10 * time.Second} |
| + if err := srv.Serve(ln); err != nil { |
| + die("hooks run: %v", err) |
| + } |
| +} |
| + |
| +// runAddHooks: `lchat add-hooks <host>` — stand up the webhook-ingest sidecar |
| +// for an instance on the router box. Mirrors add-bot: mint the service |
| +// identity's agent invite via the instance's admin API, stage the |
| +// EnvironmentFile, then (the part add-bot doesn't have) register the |
| +// `in.<host>` registry route so the router fronts the sidecar's socket, and |
| +// start the unit. Idempotent by env-file existence. |
| +func runAddHooks(args []string) { |
| + fs := flag.NewFlagSet("add-hooks", flag.ExitOnError) |
| + registryPath := fs.String("registry", envOr("LCHAT_REGISTRY", "/data/registry.db"), "router registry SQLite path") |
| + unit := fs.String("unit", envOr("LCHAT_HOOKS_UNIT", "lchat-hooks@"), "systemd hooks unit template prefix") |
| + _ = fs.Parse(reorderFlagsFirst(fs, args)) |
| + |
| + host := strings.ToLower(strings.TrimSpace(fs.Arg(0))) |
| + if !validHost(host) { |
| + die("add-hooks: invalid host %q", host) |
| + } |
| + reg, err := openRegistry(*registryPath) |
| + if err != nil { |
| + die("add-hooks: open registry: %v", err) |
| + } |
| + rt, err := reg.get(host) |
| + if err != nil { |
| + die("add-hooks: %v", err) |
| + } |
| + if rt == nil || rt.Kind != "instance" { |
| + die("add-hooks: %q is not a messenger instance (run add-instance first)", host) |
| + } |
| + |
| + envPath := hooksEnvPath(host) |
| + if _, err := os.Stat(envPath); err == nil { |
| + fmt.Printf("add-hooks: %s already staged (%s) — leaving it as is\n", host, envPath) |
| + } else { |
| + env, err := readEnvFile(filepath.Join(instanceDataDir, host+".env")) |
| + if err != nil { |
| + die("add-hooks: read instance env: %v", err) |
| + } |
| + admin := env["LCHAT_ADMIN_TOKEN"] |
| + if admin == "" { |
| + die("add-hooks: no LCHAT_ADMIN_TOKEN for %s", host) |
| + } |
| + invite, err := mintHooksServiceInvite(botMintBase(host), admin) |
| + if err != nil { |
| + die("add-hooks: mint service invite: %v", err) |
| + } |
| + sessionDir := hooksSessionDir(host) |
| + if err := os.MkdirAll(sessionDir, 0o750); err != nil { |
| + die("add-hooks: mkdir %s: %v", sessionDir, err) |
| + } |
| + if err := writeEnvFile(envPath, map[string]string{ |
| + "LCHAT_BASE": "https://" + host, |
| + "LCHAT_INVITE": invite, |
| + "LCHAT_SESSION": filepath.Join(sessionDir, "session.json"), |
| + "LCHAT_SOCKET": hooksSocketPath(host), |
| + }); err != nil { |
| + die("add-hooks: write env: %v", err) |
| + } |
| + chownToService(sessionDir) |
| + chownToService(envPath) |
| + } |
| + |
| + // The ingest host: a plain registry route — the router proxies it like any |
| + // other host, and per-host autocert mints in.<host>'s certificate on the |
| + // first request (the *.domain wildcard cert only covers one label, so this |
| + // host takes the per-host issuance path; wildcard DNS resolves it with no |
| + // zone change). A custom-domain instance needs its owner to add an |
| + // in.<host> DNS record — see docs/webhooks.md. |
| + ingest := "in." + host |
| + if err := reg.upsert(Route{Host: ingest, Kind: "hooks", Socket: hooksSocketPath(host), Status: "ready"}); err != nil { |
| + die("add-hooks: register %s: %v", ingest, err) |
| + } |
| + |
| + if err := systemctl("enable", "--now", *unit+host); err != nil { |
| + die("add-hooks: start %s%s: %v (env staged at %s; route registered)", *unit, host, err, envPath) |
| + } |
| + fmt.Printf("added webhook ingest for %s\n", host) |
| + fmt.Printf(" ingest host: https://%s (router picks up the route within seconds)\n", ingest) |
| + fmt.Printf(" manage webhooks from the app: admins → Webhooks\n") |
| + fmt.Printf(" logs: journalctl -u %s%s -f\n", *unit, host) |
| +} |
| + |
| +// mintHooksServiceInvite mints the service identity's agent invite — |
| +// mintAgentInvite's shape with hooks_service set, so the server records which |
| +// pubkey webhook definitions seal to. Not catalog-listed: this identity is |
| +// plumbing, not a helper members talk to. |
| +func mintHooksServiceInvite(base, admin string) (string, error) { |
| + t, err := e2ee.RandomSecret() |
| + if err != nil { |
| + return "", err |
| + } |
| + inv, err := e2ee.DeriveInvite(t) |
| + if err != nil { |
| + return "", err |
| + } |
| + body, err := json.Marshal(map[string]any{ |
| + "label": "Webhooks", |
| + "id": inv.ID, |
| + "wrapped": "", // fresh identity: the sidecar generates its own keypair |
| + "claim_hash": inv.ClaimHash, |
| + "agent": true, |
| + "hooks_service": true, |
| + "description": "Incoming-webhooks relay — turns outside services' events into encrypted posts.", |
| + }) |
| + if err != nil { |
| + return "", err |
| + } |
| + req, err := http.NewRequest("POST", base+"/api/admin/key-invites", bytes.NewReader(body)) |
| + if err != nil { |
| + return "", err |
| + } |
| + req.Header.Set("Content-Type", "application/json") |
| + req.Header.Set("Authorization", "Bearer "+admin) |
| + client := &http.Client{Timeout: 30 * time.Second} |
| + resp, err := client.Do(req) |
| + if err != nil { |
| + return "", err |
| + } |
| + defer resp.Body.Close() |
| + if resp.StatusCode != http.StatusOK { |
| + detail, _ := io.ReadAll(io.LimitReader(resp.Body, 512)) |
| + return "", fmt.Errorf("POST /api/admin/key-invites: HTTP %d %s", resp.StatusCode, strings.TrimSpace(string(detail))) |
| + } |
| + return t, nil |
| +} |
diff --git a/internal/hooks/hooks.go b/internal/hooks/hooks.go| new file mode 100644 |
| index 0000000..7e43437 |
| --- /dev/null |
| +++ b/internal/hooks/hooks.go |
| @@ -0,0 +1,471 @@ |
| +// Package hooks is the webhook-ingest sidecar (`lchat hooks run`, issue #610, |
| +// docs/webhooks.md) — the INVERSE of the bot sidecar. The bot sidecar is a |
| +// pure-outbound client that must never listen; this one exists to listen. It |
| +// serves `in.<host>` (a registry route behind the instance router) and turns |
| +// each inbound HTTP request into an encrypted typed event posted into the one |
| +// thread that webhook feeds. |
| +// |
| +// It is strictly write-only against the instance: |
| +// |
| +// - it never opens a websocket and never calls GET /api/messages — nothing |
| +// here can read a conversation; |
| +// - it never holds the instance key — webhook definitions are sealed |
| +// directly to its service identity's pubkey (e2ee.OpenSealed); |
| +// - per webhook it holds exactly that webhook's private key + device token |
| +// (from the definition), which unwrap exactly one thread's key. |
| +// |
| +// The outbound path is the Announcer's: unwrap the wrapped thread key, encrypt |
| +// the payload, POST /api/messages. Compromise of this process yields the |
| +// threads its webhooks feed — never the space's other threads, never a bot's |
| +// brain, never the Anthropic key (it has none). |
| +package hooks |
| + |
| +import ( |
| + "context" |
| + "encoding/base64" |
| + "encoding/json" |
| + "fmt" |
| + "io" |
| + "net/http" |
| + "net/url" |
| + "strings" |
| + "sync" |
| + "time" |
| + "unicode/utf8" |
| + |
| + "lchat/internal/bot" |
| + "lchat/internal/client" |
| + "lchat/internal/e2ee" |
| +) |
| + |
| +const ( |
| + // maxRawBody caps an inbound request body. The server's message-blob cap is |
| + // 64K; after JSON framing + encryption + base64 a 40K raw body is the safe |
| + // bound. Oversize is rejected with 413 — never silently truncated (#610). |
| + maxRawBody = 40 * 1024 |
| + // Per-webhook rate limit: a token bucket refilled 1/s with a 60-deep burst. |
| + // The server's content-blind message brake still backstops this. |
| + rateBurst = 60 |
| + ratePerSec = 1 |
| + // DefaultPoll is how often the sidecar re-reads its definitions. |
| + DefaultPoll = 10 * time.Second |
| +) |
| + |
| +// headerAllowlist is the ONLY request-header subset captured into events. |
| +// Full headers routinely carry sender credentials (Authorization, cookies, |
| +// bespoke API keys) that must never enter the log — even encrypted, every |
| +// thread member could read them. Signature headers are fine: an HMAC over one |
| +// body is evidence, not a reusable secret. |
| +var headerAllowlist = []string{ |
| + "Content-Type", |
| + "User-Agent", |
| + "X-Github-Event", |
| + "X-Github-Delivery", |
| + "X-Github-Hook-Id", |
| + "X-Gitlab-Event", |
| + "X-Gitlab-Instance", |
| + "X-Hub-Signature-256", |
| + "X-Event-Key", |
| + "X-Request-Id", |
| + "X-Slack-Signature", |
| + "X-Slack-Request-Timestamp", |
| + "X-Webhook-Event", |
| +} |
| + |
| +// Definition is one webhook, as sealed into def_blob by the creating admin's |
| +// browser (webhooks.js buildDefinition — the two must stay in step). |
| +type Definition struct { |
| + V int `json:"v"` |
| + ID string `json:"id"` // webhook row id (wh-…) |
| + UserID string `json:"user_id"` // the webhook's agent user (event attribution) |
| + Name string `json:"name"` // display name (rides inside each event) |
| + NS string `json:"ns"` // event namespace → type "webhooks.<ns>" |
| + ThreadID string `json:"thread_id"` // the ONE thread this webhook feeds |
| + Secret string `json:"secret"` // URL capability: /h/<secret> |
| + Token string `json:"token"` // the webhook's device token (posting) |
| + Priv string `json:"priv"` // the webhook's private key (thread-key unwrap) |
| +} |
| + |
| +// route is one live webhook: its definition plus delivery state that survives |
| +// definition re-polls (the unwrapped thread key and the rate bucket). |
| +type route struct { |
| + def Definition |
| + |
| + mu sync.Mutex |
| + threadKey []byte // nil until first delivery fetches + unwraps it |
| + tokens float64 // rate-limit bucket |
| + lastFill time.Time |
| +} |
| + |
| +// Sidecar is the running ingest process: an HTTP handler over a route table the |
| +// definition poll rebuilds. |
| +type Sidecar struct { |
| + base string |
| + sess *client.Session |
| + logf func(string, ...any) |
| + // Two clients, split by hibernation semantics (bot.SidecarHeader): |
| + // polling is MARKED (recurring keepalive-ish traffic must never hold a |
| + // hibernating instance awake — it just 503s while the instance sleeps and |
| + // the stale route table keeps serving), while delivery is UNMARKED — an |
| + // inbound webhook is real external activity and must WAKE a sleeping |
| + // instance to land its event. |
| + poll *http.Client |
| + post *http.Client |
| + |
| + mu sync.RWMutex |
| + routes map[string]*route // secret → route |
| + polled bool // one successful poll has filled the table |
| + warned map[string]bool // def ids we've already logged as unreadable |
| +} |
| + |
| +// New builds a sidecar for base (the instance origin) with the provisioned |
| +// service-identity session. |
| +func New(base string, sess *client.Session, logf func(string, ...any)) *Sidecar { |
| + if logf == nil { |
| + logf = func(string, ...any) {} |
| + } |
| + return &Sidecar{ |
| + base: strings.TrimRight(base, "/"), |
| + sess: sess, |
| + logf: logf, |
| + poll: &http.Client{Timeout: 30 * time.Second, Transport: markTransport{}}, |
| + post: &http.Client{Timeout: 30 * time.Second}, |
| + routes: map[string]*route{}, |
| + warned: map[string]bool{}, |
| + } |
| +} |
| + |
| +type markTransport struct{} |
| + |
| +func (markTransport) RoundTrip(req *http.Request) (*http.Response, error) { |
| + req = req.Clone(req.Context()) |
| + req.Header.Set(bot.SidecarHeader, "1") |
| + return http.DefaultTransport.RoundTrip(req) |
| +} |
| + |
| +// Run polls definitions until ctx ends. The HTTP listener is the caller's |
| +// (hookscmd wires Handler() into its server); Run only keeps the table fresh. |
| +func (s *Sidecar) Run(ctx context.Context, every time.Duration) { |
| + if every <= 0 { |
| + every = DefaultPoll |
| + } |
| + t := time.NewTicker(every) |
| + defer t.Stop() |
| + for { |
| + if err := s.PollOnce(); err != nil { |
| + s.logf("hooks: definition poll: %v", err) |
| + } |
| + select { |
| + case <-ctx.Done(): |
| + return |
| + case <-t.C: |
| + } |
| + } |
| +} |
| + |
| +// PollOnce re-reads the definition registry, rebuilds the route table (carrying |
| +// over per-route delivery state), and acks any not-yet-deployed webhook. |
| +func (s *Sidecar) PollOnce() error { |
| + // A restarted sidecar has an EMPTY table until a poll succeeds — a |
| + // delivery in that window would 404 the sender. So the very first poll is |
| + // unmarked (it may wake a sleeping instance, once, at boot); the recurring |
| + // ones are marked and simply fail while the instance sleeps, leaving the |
| + // last-known table serving. |
| + s.mu.RLock() |
| + c := s.poll |
| + if !s.polled { |
| + c = s.post |
| + } |
| + s.mu.RUnlock() |
| + resp, err := c.Get(s.base + "/api/hooks/defs?token=" + url.QueryEscape(s.sess.Token)) |
| + if err != nil { |
| + return err |
| + } |
| + defer resp.Body.Close() |
| + if resp.StatusCode != http.StatusOK { |
| + return fmt.Errorf("GET /api/hooks/defs: HTTP %d", resp.StatusCode) |
| + } |
| + var body struct { |
| + Hooks []struct { |
| + ID string `json:"id"` |
| + DefBlob string `json:"def_blob"` |
| + DeployedAt int64 `json:"deployed_at"` |
| + } `json:"hooks"` |
| + } |
| + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { |
| + return err |
| + } |
| + |
| + next := make(map[string]*route, len(body.Hooks)) |
| + var ack []string |
| + s.mu.RLock() |
| + prev := s.routes |
| + s.mu.RUnlock() |
| + prevByID := make(map[string]*route, len(prev)) |
| + for _, rt := range prev { |
| + prevByID[rt.def.ID] = rt |
| + } |
| + for _, h := range body.Hooks { |
| + var def Definition |
| + if err := e2ee.OpenSealed(h.DefBlob, s.sess.UserPriv, &def); err != nil { |
| + // Sealed to a different service identity (e.g. re-provisioned |
| + // sidecar) or corrupt — fail closed per row, loudly but once. |
| + if !s.warned[h.ID] { |
| + s.logf("hooks: definition %s is unreadable (sealed to another identity?): %v", h.ID, err) |
| + } |
| + s.warned[h.ID] = true |
| + continue |
| + } |
| + if def.Secret == "" || def.ThreadID == "" || def.Token == "" || def.Priv == "" || !validNS(def.NS) { |
| + if !s.warned[h.ID] { |
| + s.logf("hooks: definition %s is malformed — skipping", h.ID) |
| + } |
| + s.warned[h.ID] = true |
| + continue |
| + } |
| + rt := prevByID[def.ID] |
| + if rt == nil || rt.def != def { |
| + carried := &route{def: def, tokens: rateBurst, lastFill: time.Now()} |
| + if rt != nil { |
| + // Same webhook, changed definition: keep the bucket, drop the |
| + // cached key only if the thread or identity changed. |
| + rt.mu.Lock() |
| + carried.tokens, carried.lastFill = rt.tokens, rt.lastFill |
| + if rt.def.ThreadID == def.ThreadID && rt.def.Priv == def.Priv { |
| + carried.threadKey = rt.threadKey |
| + } |
| + rt.mu.Unlock() |
| + } |
| + rt = carried |
| + } |
| + next[def.Secret] = rt |
| + if h.DeployedAt == 0 { |
| + ack = append(ack, def.ID) |
| + } |
| + } |
| + s.mu.Lock() |
| + s.routes = next |
| + s.polled = true |
| + s.mu.Unlock() |
| + |
| + for _, id := range ack { |
| + if err := s.ackDeployed(id); err != nil { |
| + s.logf("hooks: deploy ack %s: %v", id, err) |
| + } else { |
| + s.logf("hooks: webhook %s deployed", id) |
| + } |
| + } |
| + return nil |
| +} |
| + |
| +func (s *Sidecar) ackDeployed(id string) error { |
| + raw, _ := json.Marshal(map[string]string{"token": s.sess.Token}) |
| + resp, err := s.poll.Post(s.base+"/api/hooks/"+url.PathEscape(id)+"/deployed", "application/json", strings.NewReader(string(raw))) |
| + if err != nil { |
| + return err |
| + } |
| + defer resp.Body.Close() |
| + if resp.StatusCode != http.StatusOK { |
| + return fmt.Errorf("HTTP %d", resp.StatusCode) |
| + } |
| + return nil |
| +} |
| + |
| +// Routes reports how many webhooks are currently live (logging/tests). |
| +func (s *Sidecar) Routes() int { |
| + s.mu.RLock() |
| + defer s.mu.RUnlock() |
| + return len(s.routes) |
| +} |
| + |
| +// Handler is the ingest surface: ANY method on /h/<secret>[/<subpath>]. |
| +// Unknown secrets 404 — the sidecar alone knows which webhooks exist; the |
| +// router forwards blindly. |
| +func (s *Sidecar) Handler() http.Handler { |
| + mux := http.NewServeMux() |
| + mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) { |
| + fmt.Fprintln(w, "ok") |
| + }) |
| + mux.HandleFunc("/h/", s.serveHook) |
| + mux.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) { |
| + http.Error(w, "not found", http.StatusNotFound) |
| + }) |
| + return mux |
| +} |
| + |
| +func (s *Sidecar) serveHook(w http.ResponseWriter, r *http.Request) { |
| + rest := strings.TrimPrefix(r.URL.Path, "/h/") |
| + secret, subpath, _ := strings.Cut(rest, "/") |
| + s.mu.RLock() |
| + rt := s.routes[secret] |
| + s.mu.RUnlock() |
| + if secret == "" || rt == nil { |
| + http.Error(w, "not found", http.StatusNotFound) |
| + return |
| + } |
| + if !rt.allow() { |
| + http.Error(w, "rate limited", http.StatusTooManyRequests) |
| + return |
| + } |
| + // Read the capped body; MaxBytesReader makes an over-cap send a hard error |
| + // rather than a silent prefix. |
| + raw, err := io.ReadAll(http.MaxBytesReader(w, r.Body, maxRawBody)) |
| + if err != nil { |
| + http.Error(w, "body too large", http.StatusRequestEntityTooLarge) |
| + return |
| + } |
| + |
| + event := s.buildEvent(rt.def, r, subpath, raw) |
| + if err := s.deliver(rt, event); err != nil { |
| + s.logf("hooks: deliver %s: %v", rt.def.ID, err) |
| + http.Error(w, "delivery failed", http.StatusBadGateway) |
| + return |
| + } |
| + writeJSON(w, map[string]any{"ok": true}) |
| +} |
| + |
| +// buildEvent shapes the typed event. Dumb ingest, smart render: the raw request |
| +// rides inside the event forever, so client-side rendering can improve later |
| +// without re-ingesting. Pinned by test/event-vectors/webhooks.json. |
| +func (s *Sidecar) buildEvent(def Definition, r *http.Request, subpath string, raw []byte) map[string]any { |
| + headers := map[string]string{} |
| + for _, h := range headerAllowlist { |
| + if v := r.Header.Get(h); v != "" { |
| + headers[h] = v |
| + } |
| + } |
| + query := map[string][]string(r.URL.Query()) |
| + body, b64 := string(raw), false |
| + if !utf8.ValidString(body) { |
| + body, b64 = base64.RawURLEncoding.EncodeToString(raw), true |
| + } |
| + req := map[string]any{ |
| + "method": r.Method, |
| + "path": "/" + subpath, // after the secret segment; the secret itself never enters the log |
| + "query": query, |
| + "headers": headers, |
| + "body": body, |
| + } |
| + if b64 { |
| + req["body_b64"] = true |
| + } |
| + return map[string]any{ |
| + "type": "webhooks." + def.NS, |
| + "name": def.Name, |
| + "uid": def.UserID, |
| + "ts": time.Now().UnixMilli(), |
| + "hook": map[string]any{"id": def.ID, "ns": def.NS}, |
| + "req": req, |
| + } |
| +} |
| + |
| +// deliver encrypts the event to the webhook's thread key and posts it — the |
| +// Announcer's outbound-only walk (unwrap → EncryptJSON → POST /api/messages). |
| +func (s *Sidecar) deliver(rt *route, event map[string]any) error { |
| + key, err := s.threadKey(rt) |
| + if err != nil { |
| + return err |
| + } |
| + blob, err := e2ee.EncryptJSON(key, event) |
| + if err != nil { |
| + return err |
| + } |
| + raw, _ := json.Marshal(map[string]string{"thread_id": rt.def.ThreadID, "blob": blob}) |
| + resp, err := s.post.Post(s.base+"/api/messages?token="+url.QueryEscape(rt.def.Token), "application/json", strings.NewReader(string(raw))) |
| + if err != nil { |
| + return err |
| + } |
| + defer resp.Body.Close() |
| + if resp.StatusCode != http.StatusOK { |
| + return fmt.Errorf("POST /api/messages: HTTP %d", resp.StatusCode) |
| + } |
| + return nil |
| +} |
| + |
| +// threadKey returns the webhook's unwrapped thread key, fetching it on first |
| +// use: GET /api/threads under the WEBHOOK'S OWN token lists exactly the one |
| +// thread it was added to, with the key wrapped to its pubkey at creation. |
| +func (s *Sidecar) threadKey(rt *route) ([]byte, error) { |
| + rt.mu.Lock() |
| + cached := rt.threadKey |
| + rt.mu.Unlock() |
| + if cached != nil { |
| + return cached, nil |
| + } |
| + // Unmarked like deliver: this fetch happens mid-delivery, so it too must |
| + // be able to wake a sleeping instance. |
| + resp, err := s.post.Get(s.base + "/api/threads?token=" + url.QueryEscape(rt.def.Token)) |
| + if err != nil { |
| + return nil, err |
| + } |
| + defer resp.Body.Close() |
| + if resp.StatusCode != http.StatusOK { |
| + return nil, fmt.Errorf("GET /api/threads: HTTP %d", resp.StatusCode) |
| + } |
| + var body struct { |
| + Threads []struct { |
| + ID string `json:"id"` |
| + Wrapped string `json:"wrapped"` |
| + } `json:"threads"` |
| + } |
| + if err := json.NewDecoder(resp.Body).Decode(&body); err != nil { |
| + return nil, err |
| + } |
| + for _, t := range body.Threads { |
| + if t.ID != rt.def.ThreadID || t.Wrapped == "" { |
| + continue |
| + } |
| + keyB64, err := e2ee.UnwrapThreadKey(t.Wrapped, rt.def.Priv) |
| + if err != nil { |
| + return nil, fmt.Errorf("unwrap thread key: %w", err) |
| + } |
| + key, err := e2ee.ImportKey(keyB64) |
| + if err != nil { |
| + return nil, err |
| + } |
| + rt.mu.Lock() |
| + rt.threadKey = key |
| + rt.mu.Unlock() |
| + return key, nil |
| + } |
| + return nil, fmt.Errorf("thread %s not visible to webhook %s (membership missing?)", rt.def.ThreadID, rt.def.ID) |
| +} |
| + |
| +// allow is the per-webhook token bucket. |
| +func (rt *route) allow() bool { |
| + rt.mu.Lock() |
| + defer rt.mu.Unlock() |
| + now := time.Now() |
| + rt.tokens += now.Sub(rt.lastFill).Seconds() * ratePerSec |
| + if rt.tokens > rateBurst { |
| + rt.tokens = rateBurst |
| + } |
| + rt.lastFill = now |
| + if rt.tokens < 1 { |
| + return false |
| + } |
| + rt.tokens-- |
| + return true |
| +} |
| + |
| +// validNS mirrors the client-side namespace rule: lowercase slug, so the event |
| +// type "webhooks.<ns>" is always a well-formed app event type. |
| +func validNS(ns string) bool { |
| + if ns == "" || len(ns) > 32 { |
| + return false |
| + } |
| + for i, c := range ns { |
| + switch { |
| + case c >= 'a' && c <= 'z', c >= '0' && c <= '9': |
| + case c == '-' && i > 0: |
| + default: |
| + return false |
| + } |
| + } |
| + return true |
| +} |
| + |
| +func writeJSON(w http.ResponseWriter, v any) { |
| + w.Header().Set("Content-Type", "application/json") |
| + _ = json.NewEncoder(w).Encode(v) |
| +} |
diff --git a/ios/LChatCore/Tests/LChatCoreTests/Fixtures/webhooks-vector.json b/ios/LChatCore/Tests/LChatCoreTests/Fixtures/webhooks-vector.json| new file mode 100644 |
| index 0000000..35cd4fc |
| --- /dev/null |
| +++ b/ios/LChatCore/Tests/LChatCoreTests/Fixtures/webhooks-vector.json |
| @@ -0,0 +1,44 @@ |
| +{ |
| + "description": "Incoming-webhook events (issue #610, docs/webhooks.md) — native's half of the ignore-unclaimed-types contract. Twin of test/event-vectors/webhooks.json; keep the event payloads in step.", |
| + "vectors": [ |
| + { |
| + "name": "github-workflow-run", |
| + "event": { |
| + "type": "webhooks.github", |
| + "name": "CI runs", |
| + "uid": "hookuser123", |
| + "ts": 1752710400000, |
| + "hook": { "id": "wh-abc123", "ns": "github" }, |
| + "req": { |
| + "method": "POST", |
| + "path": "/", |
| + "query": {}, |
| + "headers": { |
| + "Content-Type": "application/json", |
| + "X-Github-Event": "workflow_run", |
| + "X-Github-Delivery": "72d3162e-cc78-11e3-81ab-4c9367dc0958" |
| + }, |
| + "body": "{\"action\":\"completed\",\"workflow_run\":{\"status\":\"completed\",\"conclusion\":\"success\"}}" |
| + } |
| + } |
| + }, |
| + { |
| + "name": "binary-body-custom-namespace", |
| + "event": { |
| + "type": "webhooks.sensor-net", |
| + "name": "Greenhouse", |
| + "uid": "hookuser456", |
| + "ts": 1752710500000, |
| + "hook": { "id": "wh-def456", "ns": "sensor-net" }, |
| + "req": { |
| + "method": "PUT", |
| + "path": "/readings", |
| + "query": { "site": ["7"] }, |
| + "headers": { "Content-Type": "application/octet-stream" }, |
| + "body": "AAECAwQF", |
| + "body_b64": true |
| + } |
| + } |
| + } |
| + ] |
| +} |
diff --git a/ios/LChatCore/Tests/LChatCoreTests/WebhooksVectorTests.swift b/ios/LChatCore/Tests/LChatCoreTests/WebhooksVectorTests.swift| new file mode 100644 |
| index 0000000..ee6a54f |
| --- /dev/null |
| +++ b/ios/LChatCore/Tests/LChatCoreTests/WebhooksVectorTests.swift |
| @@ -0,0 +1,40 @@ |
| +import XCTest |
| +@testable import LChatCore |
| + |
| +// 🤖 Incoming-webhook events (issue #610, docs/webhooks.md): the hooks sidecar |
| +// posts typed `webhooks.<ns>` payloads into ordinary threads. Native claims |
| +// none of them — MessagePayload.isRenderableChat is the gate that keeps |
| +// unclaimed typed events out of the feed and out of notifications, exactly as |
| +// the web's foldEvent/isAppType gate does (test/event-vectors/webhooks.json is |
| +// the twin of this fixture; keep the payloads in step). These vectors pin two |
| +// things: the payload DECODES tolerantly (the unknown hook/req envelope never |
| +// fails MessagePayload), and it stays non-renderable. |
| +final class WebhooksVectorTests: XCTestCase { |
| + func testWebhookEventsDecodeAndStayNonRenderable() throws { |
| + let url = try XCTUnwrap( |
| + Bundle.module.url(forResource: "Fixtures/webhooks-vector", withExtension: "json"), |
| + "missing fixture webhooks-vector.json") |
| + let raw = try JSONSerialization.jsonObject(with: Data(contentsOf: url)) as? [String: Any] |
| + let vectors = try XCTUnwrap(raw?["vectors"] as? [[String: Any]], "fixture has no vectors") |
| + XCTAssertFalse(vectors.isEmpty) |
| + |
| + for vector in vectors { |
| + let name = vector["name"] as? String ?? "?" |
| + let event = try XCTUnwrap(vector["event"] as? [String: Any], "\(name): no event") |
| + let data = try JSONSerialization.data(withJSONObject: event) |
| + // Tolerant decode: the hook/req envelope is unknown to native and |
| + // must be ignored, never a decode failure. |
| + let p = try JSONDecoder().decode(MessagePayload.self, from: data) |
| + |
| + XCTAssertEqual(p.type, event["type"] as? String, "\(name): type must round-trip") |
| + XCTAssertEqual(p.name, event["name"] as? String, "\(name): attribution name") |
| + XCTAssertEqual(p.uid, event["uid"] as? String, "\(name): attribution uid") |
| + // The contract: an unclaimed typed event renders nothing — no |
| + // bubble, no notification. A future consumer surface may claim a |
| + // webhooks.* type deliberately; flipping this assertion then must |
| + // be a conscious cross-platform change with the web's gate. |
| + XCTAssertFalse(p.isRenderableChat, "\(name): webhook events must not render as chat") |
| + XCTAssertFalse(p.isReactionEvent, "\(name): webhook events are not reactions") |
| + } |
| + } |
| +} |
diff --git a/main.go b/main.go| index b9cd29d..faa236c 100644 |
| --- a/main.go |
| +++ b/main.go |
| @@ -176,6 +176,12 @@ func main() { |
| case "bot": |
| runBotCmd(os.Args[2:]) |
| return |
| + case "add-hooks": |
| + runAddHooks(os.Args[2:]) |
| + return |
| + case "hooks": |
| + runHooksCmd(os.Args[2:]) |
| + return |
| case "links": |
| runLinksCmd(os.Args[2:]) |
| return |
| @@ -449,6 +455,16 @@ func (s *Server) handler() http.Handler { |
| // GET /api/next-due so hibernated charters still fire on time). |
| mux.HandleFunc("POST /api/charter-next-due", s.postCharterNextDue) |
| mux.HandleFunc("POST /api/bot-usage", s.postBotUsage) |
| + // Incoming webhooks (hooks.go, docs/webhooks.md): admin-managed registry of |
| + // opaque definition blobs the hooks sidecar polls; deployed is the sidecar's |
| + // plaintext go-live ack. |
| + mux.HandleFunc("GET /api/hooks/service", s.getHooksService) |
| + mux.HandleFunc("POST /api/hooks", s.createWebhookHandler) |
| + mux.HandleFunc("PUT /api/hooks/{id}/blobs", s.putWebhookBlobsHandler) |
| + mux.HandleFunc("GET /api/hooks", s.listWebhooksHandler) |
| + mux.HandleFunc("GET /api/hooks/defs", s.listWebhookDefsHandler) |
| + mux.HandleFunc("POST /api/hooks/{id}/deployed", s.webhookDeployedHandler) |
| + mux.HandleFunc("DELETE /api/hooks/{id}", s.deleteWebhookHandler) |
| // The sandbox frame document: an opaque-origin cage for member-app code. |
| mux.HandleFunc("GET /app-sandbox", s.serveAppSandbox) |
| @@ -3164,6 +3180,11 @@ func (s *Server) createKeyInvite(w http.ResponseWriter, r *http.Request) { |
| // add-instance for a fresh instance's first user (and available to admins |
| // for promoting someone at invite time). See docs/conversational-onboarding.md. |
| GrantsAdmin bool `json:"grants_admin"` |
| + // HooksService marks the hooks sidecar's service identity invite |
| + // (add-hooks mints it — hooks.go, docs/webhooks.md): the claimer is |
| + // recorded in config so webhook creation knows which pubkey to seal |
| + // definitions to. Not an AI bot, so it bypasses the bots gate. |
| + HooksService bool `json:"hooks_service"` |
| // TierID makes this a COHORT invite: the claimer is assigned that |
| // access level on first-device claim ('invite' source). Admin-minted |
| // only, which is what keeps level assignment an admin power. |
| @@ -3183,8 +3204,10 @@ func (s *Server) createKeyInvite(w http.ResponseWriter, r *http.Request) { |
| } |
| // The bots gate stops NEW bot identities being minted (checked before the |
| // user row exists, so a rejected mint leaves nothing behind); bots already |
| - // on the roster keep running — the admin retires those by removal. |
| - if body.Agent && !s.db.botsEnabled() { |
| + // on the roster keep running — the admin retires those by removal. The |
| + // hooks service identity is exempt: it's integration plumbing (no AI, no |
| + // conversations), gated instead by webhooks being admin-only. |
| + if body.Agent && !body.HooksService && !s.db.botsEnabled() { |
| httpError(w, http.StatusForbidden, "bots are turned off for this space") |
| return |
| } |
| @@ -3220,6 +3243,20 @@ func (s *Server) createKeyInvite(w http.ResponseWriter, r *http.Request) { |
| httpError(w, http.StatusInternalServerError, "db error") |
| return |
| } |
| + if body.HooksService { |
| + // Record which identity webhook definitions seal to, and badge it |
| + // distinctly from conversational bots. Last mint wins — re-running |
| + // add-hooks after a lost session repoints creation at the fresh |
| + // identity (stale defs sealed to the old key simply stop deploying). |
| + if err := s.db.setConfig(hooksServiceUserKey, userID); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + if err := s.db.setBotKind(userID, "webhook"); err != nil { |
| + httpError(w, http.StatusInternalServerError, "db error") |
| + return |
| + } |
| + } |
| } |
| if body.TierID != "" && !s.db.tierExists(body.TierID) { |
| httpError(w, http.StatusBadRequest, "bad access level") |
diff --git a/test/event-vectors.test.mjs b/test/event-vectors.test.mjs| index adac15b..3666e79 100644 |
| --- a/test/event-vectors.test.mjs |
| +++ b/test/event-vectors.test.mjs |
| @@ -11,6 +11,10 @@ import { viewsApp } from "../web/static/apps/views.js"; |
| // cross-IMPLEMENTATION contract with the Go fold (internal/vault), which reads |
| // the same test/event-vectors/vault.json in its own test suite. |
| import { app as vaultApp } from "../web/static/apps/vault.js"; |
| +// 🤖 Webhooks: the ignore-unclaimed-types contract runs against the REAL |
| +// dispatch gate, with the shipped webhooks app loaded (it must claim nothing). |
| +import { isAppType, foldEvent } from "../web/static/apps.js"; |
| +import "../web/static/apps/webhooks.js"; |
| // Import chat.js fold handlers — the ones we're testing. |
| // The app.hostEvents from web/static/apps/chat.js. |
| @@ -562,6 +566,22 @@ allVectors.views.forEach((vector) => { |
| }); |
| }); |
| +// 🤖 Webhook vectors (docs/webhooks.md, #610) — the contract here is the |
| +// ABSENCE of handling: webhooks.<ns> events are unclaimed until a consumer app |
| +// registers the exact type, and unclaimed typed events must be silently |
| +// ignored. Tested against the REAL dispatch gate (isAppType/foldEvent from |
| +// apps.js) with the shipped webhooks management app loaded, so a future change |
| +// that accidentally claims (or crashes on) these types fails here. Native pins |
| +// the same contract in WebhooksVectorTests.swift (MessagePayload.isRenderableChat). |
| +allVectors.webhooks.forEach((vector) => { |
| + test(`webhooks: ${vector.name} — unclaimed type is silently ignored`, () => { |
| + assert.strictEqual(vector.expected, "ignored", "webhook vectors document the ignore contract"); |
| + assert.strictEqual(isAppType(vector.event.type), false, `${vector.event.type} must be unclaimed (no shipped app owns webhooks.* types)`); |
| + const folded = foldEvent("thread-1", vector.event, { id: 1, ts: vector.event.ts }); |
| + assert.strictEqual(folded, null, "foldEvent must drop an unclaimed type without error"); |
| + }); |
| +}); |
| + |
| // 🤖 Vault vectors (docs/locked-vaults.md) — folded by the real reducer and |
| // compared in CANONICAL form (every field present, ""/0 defaults, keyring null |
| // when unlocked), the same normalisation the Go side applies, so web and CLI |
diff --git a/test/event-vectors/index.js b/test/event-vectors/index.js| index c3e8590..b08e840 100644 |
| --- a/test/event-vectors/index.js |
| +++ b/test/event-vectors/index.js |
| @@ -12,6 +12,7 @@ export const spaceAppsVectors = JSON.parse(readFileSync(join(__dirname, "spaceap |
| export const adminVectors = JSON.parse(readFileSync(join(__dirname, "admin.json"), "utf8")); |
| export const viewsVectors = JSON.parse(readFileSync(join(__dirname, "views.json"), "utf8")); |
| export const vaultVectors = JSON.parse(readFileSync(join(__dirname, "vault.json"), "utf8")); |
| +export const webhooksVectors = JSON.parse(readFileSync(join(__dirname, "webhooks.json"), "utf8")); |
| // Flatten all vectors for easy iteration in tests |
| export const allVectors = { |
| @@ -23,4 +24,5 @@ export const allVectors = { |
| admin: adminVectors.vectors || [], |
| views: viewsVectors.vectors || [], |
| vault: vaultVectors.vectors || [], |
| + webhooks: webhooksVectors.vectors || [], |
| }; |
diff --git a/test/event-vectors/webhooks.json b/test/event-vectors/webhooks.json| new file mode 100644 |
| index 0000000..491403b |
| --- /dev/null |
| +++ b/test/event-vectors/webhooks.json |
| @@ -0,0 +1,59 @@ |
| +{ |
| + "description": "Incoming-webhook events (issue #610, docs/webhooks.md): the shape the hooks sidecar emits, and the platform contract that UNCLAIMED typed events are silently ignored on every client.", |
| + "app": "webhooks", |
| + "source": "internal/hooks/hooks.go buildEvent (emitter); web/static/apps.js foldEvent/isAppType (web ignore gate); ios MessagePayload.isRenderableChat (native ignore gate)", |
| + "state_shape": "none — the webhooks management app deliberately claims NO wire types; webhooks.<ns> events belong to future CONSUMER apps (e.g. a CI dashboard claiming webhooks.github)", |
| + "notes": [ |
| + "The sidecar posts one event per inbound HTTP request, encrypted to the target thread's key: { type: 'webhooks.<ns>', name, uid, ts, hook: {id, ns}, req: {method, path, query, headers, body[, body_b64]} }.", |
| + "name/uid are the webhook's roster identity — the same attribution fields as chat messages, so mute-by-person and roster labels apply.", |
| + "req.headers carries only the sidecar's allowlisted subset (Content-Type, User-Agent, provider event/signature headers); Authorization and other sender credentials never enter the log.", |
| + "req.body is the raw body string; body_b64: true marks a non-UTF-8 body carried base64url-encoded. req.path is the sub-path AFTER the secret segment — the secret itself never rides inside an event.", |
| + "CONTRACT — unclaimed types are ignored: until some app registers the exact type string in its events map, web's foldEvent returns null / isAppType is false (the message renders nothing, notifies nothing), and native's MessagePayload.isRenderableChat is false (no bubble, no notification). Old clients degrade to exactly this behaviour.", |
| + "Space-thread deliveries additionally never push (push.go skips kind='space' entirely)." |
| + ], |
| + "vectors": [ |
| + { |
| + "name": "github-workflow-run", |
| + "description": "A GitHub Actions delivery: the full event shape as the sidecar builds it. Unclaimed by any shipped app — web fold must return null, native must treat it as non-renderable.", |
| + "event": { |
| + "type": "webhooks.github", |
| + "name": "CI runs", |
| + "uid": "hookuser123", |
| + "ts": 1752710400000, |
| + "hook": { "id": "wh-abc123", "ns": "github" }, |
| + "req": { |
| + "method": "POST", |
| + "path": "/", |
| + "query": {}, |
| + "headers": { |
| + "Content-Type": "application/json", |
| + "X-Github-Event": "workflow_run", |
| + "X-Github-Delivery": "72d3162e-cc78-11e3-81ab-4c9367dc0958" |
| + }, |
| + "body": "{\"action\":\"completed\",\"workflow_run\":{\"status\":\"completed\",\"conclusion\":\"success\"}}" |
| + } |
| + }, |
| + "expected": "ignored" |
| + }, |
| + { |
| + "name": "binary-body-custom-namespace", |
| + "description": "A non-UTF-8 delivery under a member-chosen namespace: body rides base64url with body_b64 set. Same ignore contract.", |
| + "event": { |
| + "type": "webhooks.sensor-net", |
| + "name": "Greenhouse", |
| + "uid": "hookuser456", |
| + "ts": 1752710500000, |
| + "hook": { "id": "wh-def456", "ns": "sensor-net" }, |
| + "req": { |
| + "method": "PUT", |
| + "path": "/readings", |
| + "query": { "site": ["7"] }, |
| + "headers": { "Content-Type": "application/octet-stream" }, |
| + "body": "AAECAwQF", |
| + "body_b64": true |
| + } |
| + }, |
| + "expected": "ignored" |
| + } |
| + ] |
| +} |
diff --git a/updater_test.go b/updater_test.go| index 03b62d6..ed3f717 100644 |
| --- a/updater_test.go |
| +++ b/updater_test.go |
| @@ -266,19 +266,20 @@ func TestUpdatePassAppliesPerPrefs(t *testing.T) { |
| t.Fatalf("h2 registry = %s", rt.Version) |
| } |
| - // Two instances moved (h1, h4); each restarts the instance and then its bot |
| - // sidecar so the bot tracks the new version (restartBotSidecar). Four calls, |
| - // no more — h2 (download-only) and h3 (downloads off) never restart. |
| + // Two instances moved (h1, h4); each restarts the instance and then its |
| + // sidecars so both track the new version (restartBotSidecar + |
| + // restartHooksSidecar). Six calls, no more — h2 (download-only) and h3 |
| + // (downloads off) never restart. |
| got := strings.Join(*calls, "; ") |
| for _, want := range []string{ |
| - "restart lchat@h1.example.com", "try-restart lchat-bot@h1.example.com", |
| - "restart lchat@h4.example.com", "try-restart lchat-bot@h4.example.com", |
| + "restart lchat@h1.example.com", "try-restart lchat-bot@h1.example.com", "try-restart lchat-hooks@h1.example.com", |
| + "restart lchat@h4.example.com", "try-restart lchat-bot@h4.example.com", "try-restart lchat-hooks@h4.example.com", |
| } { |
| if !strings.Contains(got, want) { |
| t.Fatalf("missing %q in systemctl calls = %v", want, *calls) |
| } |
| } |
| - if len(*calls) != 4 { |
| + if len(*calls) != 6 { |
| t.Fatalf("systemctl calls = %v", *calls) |
| } |
diff --git a/web/static/apps/webhooks.js b/web/static/apps/webhooks.js| new file mode 100644 |
| index 0000000..839469b |
| --- /dev/null |
| +++ b/web/static/apps/webhooks.js |
| @@ -0,0 +1,297 @@ |
| +// The Webhooks app — admin management surface for incoming webhooks (#610, |
| +// docs/webhooks.md). Admin-only and host-owned like the Admin app: webhooks |
| +// are real server state, not thread events, so everything reads and writes |
| +// through the host bridge (setSetupBridge in shell.js) — the app itself holds |
| +// no tokens, no keys, and no fetch calls of its own. |
| +// |
| +// What a webhook IS: an agent identity that turns inbound HTTP on |
| +// https://in.<host>/h/<secret> into encrypted `webhooks.<ns>` events posted |
| +// into ONE thread. This app creates them (name + event namespace + target |
| +// thread), lists them with their URLs and deploy state, and removes them. |
| +// |
| +// Deliberately, this app claims NO wire event types: `webhooks.<ns>` events |
| +// belong to the CONSUMER apps that render them (a CI dashboard claiming |
| +// "webhooks.github", say). Until such an app claims a type, its events are |
| +// silently ignored — the platform contract for unclaimed types |
| +// (test/event-vectors/webhooks.json). |
| +import { defineApp, namespace, setupBridge, registerFloorCommand, openAppThread } from "../apps.js"; |
| + |
| +const N = namespace("webhooks"); |
| + |
| +// /webhooks works from any chat, gated on role exactly like /admin. |
| +registerFloorCommand("/webhooks", { |
| + desc: "Manage incoming webhooks (admins only)", |
| + run: async (_args, _threadId, reply) => { |
| + const bridge = setupBridge(); |
| + if (!bridge || !(await bridge.isAdmin())) { |
| + reply({ text: "Only this space's admins can manage webhooks." }); |
| + return; |
| + } |
| + openAppThread("webhooks"); |
| + }, |
| +}); |
| + |
| +const NS_RE = /^[a-z0-9][a-z0-9-]{0,31}$/; |
| + |
| +// One refresh timer per mounted pane, cancelled via ctx.onCleanup — polling |
| +// only runs while a not-yet-deployed webhook is on screen, to flip its |
| +// "Deploying…" chip to "Active" without a manual refresh. |
| +let refreshTimer = null; |
| + |
| +export const app = defineApp({ |
| + name: "webhooks", |
| + title: "Webhooks", |
| + glyph: "📥", |
| + places: [], // host-mounted, admin-only — never in the Apps picker for members |
| + snapshot: false, |
| + initialState: () => ({}), |
| + events: {}, // no wire types — see the header comment |
| + render(_state, el, ctx) { |
| + paint(el, ctx); |
| + }, |
| + styles: ` |
| + .${N.cls("root")} { padding: 16px; max-width: 640px; margin: 0 auto; overflow-y: auto; height: 100%; box-sizing: border-box; } |
| + .${N.cls("root")} h2 { margin: 0 0 4px; } |
| + .${N.cls("blurb")} { color: var(--muted, #777); margin: 0 0 16px; font-size: 0.92em; } |
| + .${N.cls("card")} { border: 1px solid var(--border, #ccc3); border-radius: 10px; padding: 12px; margin: 0 0 10px; } |
| + .${N.cls("card")} h3 { margin: 0 0 2px; font-size: 1em; } |
| + .${N.cls("meta")} { color: var(--muted, #777); font-size: 0.85em; margin: 0 0 8px; } |
| + .${N.cls("chip")} { display: inline-block; border: 1px solid var(--border, #ccc3); border-radius: 999px; padding: 1px 8px; font-size: 0.8em; margin-left: 6px; } |
| + .${N.cls("chip-live")} { border-color: #2e7d3266; } |
| + .${N.cls("url")} { display: flex; gap: 6px; align-items: center; } |
| + .${N.cls("url")} code { flex: 1; overflow-x: auto; white-space: nowrap; font-size: 0.82em; padding: 6px 8px; background: var(--bubble, #8881); border-radius: 6px; } |
| + .${N.cls("root")} button { min-height: 28px; border-radius: 6px; } |
| + .${N.cls("row")} { display: flex; gap: 8px; margin: 8px 0 0; flex-wrap: wrap; } |
| + .${N.cls("form")} input, .${N.cls("form")} select { padding: 6px 8px; border-radius: 6px; border: 1px solid var(--border, #ccc4); min-width: 0; } |
| + .${N.cls("form")} label { display: block; font-size: 0.85em; margin: 10px 0 2px; } |
| + .${N.cls("note")} { font-size: 0.85em; margin: 8px 0 0; } |
| + .${N.cls("note-err")} { color: #c62828; } |
| + .${N.cls("note-ok")} { color: #2e7d32; } |
| + `, |
| +}); |
| + |
| +async function paint(el, ctx) { |
| + if (refreshTimer) { |
| + clearTimeout(refreshTimer); |
| + refreshTimer = null; |
| + } |
| + el.textContent = ""; |
| + const root = document.createElement("div"); |
| + root.className = N.cls("root"); |
| + el.appendChild(root); |
| + |
| + const h = document.createElement("h2"); |
| + h.textContent = "Incoming webhooks"; |
| + const blurb = document.createElement("p"); |
| + blurb.className = N.cls("blurb"); |
| + blurb.textContent = |
| + "Give an outside service a URL, and every request it sends becomes an encrypted event in the thread you choose — the server never sees the content, and the receiver can never read anything back."; |
| + root.append(h, blurb); |
| + |
| + const bridge = setupBridge(); |
| + if (!bridge || !bridge.hooksList) return; |
| + if (!(await bridge.isAdmin())) { |
| + root.appendChild(para("Only this space's admins can manage webhooks.")); |
| + return; |
| + } |
| + |
| + const svc = await bridge.hooksService(); |
| + if (!svc || !svc.public_key) { |
| + root.appendChild( |
| + para( |
| + "Webhooks aren't switched on for this space yet — the ingest sidecar isn't provisioned. An operator sets it up on the server box with: lchat add-hooks <host>", |
| + ), |
| + ); |
| + return; |
| + } |
| + |
| + const list = document.createElement("div"); |
| + root.appendChild(list); |
| + const rows = await bridge.hooksList(); |
| + ctx.onCleanup(() => { |
| + if (refreshTimer) clearTimeout(refreshTimer); |
| + refreshTimer = null; |
| + }); |
| + if (rows === null) { |
| + list.appendChild(para("Couldn't load webhooks — try again in a moment.")); |
| + } else if (!rows.length) { |
| + list.appendChild(para("No webhooks yet.")); |
| + } else { |
| + for (const hk of rows) list.appendChild(hookCard(hk, el, ctx)); |
| + // While any webhook is still deploying, refresh gently so the chip flips |
| + // to Active when the sidecar's next poll picks it up. |
| + if (rows.some((r) => !r.deployed) && el.isConnected) { |
| + refreshTimer = setTimeout(() => { |
| + if (el.isConnected) paint(el, ctx); |
| + }, 5000); |
| + } |
| + } |
| + |
| + root.appendChild(createForm(el, ctx, bridge)); |
| +} |
| + |
| +function para(text) { |
| + const p = document.createElement("p"); |
| + p.className = N.cls("blurb"); |
| + p.textContent = text; |
| + return p; |
| +} |
| + |
| +function hookCard(hk, el, ctx) { |
| + const card = document.createElement("div"); |
| + card.className = N.cls("card"); |
| + |
| + const title = document.createElement("h3"); |
| + title.textContent = hk.name || "Unnamed webhook"; |
| + const status = document.createElement("span"); |
| + status.className = N.cls("chip") + (hk.deployed ? " " + N.cls("chip-live") : ""); |
| + status.textContent = hk.deployed ? "Active" : "Deploying…"; |
| + title.appendChild(status); |
| + |
| + const meta = document.createElement("p"); |
| + meta.className = N.cls("meta"); |
| + meta.textContent = `Events: webhooks.${hk.ns || "?"} → ${hk.target || "a chat"}`; |
| + |
| + card.append(title, meta); |
| + |
| + if (hk.url) { |
| + const urlRow = document.createElement("div"); |
| + urlRow.className = N.cls("url"); |
| + const code = document.createElement("code"); |
| + code.textContent = hk.url; |
| + const copy = document.createElement("button"); |
| + copy.type = "button"; |
| + copy.textContent = "Copy"; |
| + copy.setAttribute("aria-label", `Copy the URL for ${hk.name}`); |
| + copy.onclick = async () => { |
| + try { |
| + await navigator.clipboard.writeText(hk.url); |
| + copy.textContent = "Copied"; |
| + setTimeout(() => (copy.textContent = "Copy"), 1500); |
| + } catch { |
| + copy.textContent = "Select + copy"; |
| + } |
| + }; |
| + urlRow.append(code, copy); |
| + card.appendChild(urlRow); |
| + } |
| + |
| + const actions = document.createElement("div"); |
| + actions.className = N.cls("row"); |
| + const remove = document.createElement("button"); |
| + remove.type = "button"; |
| + remove.textContent = "Remove"; |
| + remove.setAttribute("aria-label", `Remove the webhook ${hk.name}`); |
| + let armed = false; |
| + remove.onclick = async () => { |
| + if (!armed) { |
| + // Two-tap confirm, in place: removal kills the URL and the identity. |
| + armed = true; |
| + remove.textContent = "Really remove? The URL stops working."; |
| + return; |
| + } |
| + remove.disabled = true; |
| + const ok = await setupBridge().hooksRemove(hk.id); |
| + if (ok) paint(el, ctx); |
| + else { |
| + remove.disabled = false; |
| + armed = false; |
| + remove.textContent = "Remove"; |
| + card.appendChild(note("Couldn't remove it — try again.", true)); |
| + } |
| + }; |
| + actions.appendChild(remove); |
| + card.appendChild(actions); |
| + return card; |
| +} |
| + |
| +function note(text, isErr) { |
| + const p = document.createElement("p"); |
| + p.className = N.cls("note") + " " + (isErr ? N.cls("note-err") : N.cls("note-ok")); |
| + p.textContent = text; |
| + return p; |
| +} |
| + |
| +function createForm(el, ctx, bridge) { |
| + const form = document.createElement("form"); |
| + form.className = N.cls("card") + " " + N.cls("form"); |
| + |
| + const title = document.createElement("h3"); |
| + title.textContent = "New webhook"; |
| + form.appendChild(title); |
| + |
| + const nameLabel = document.createElement("label"); |
| + nameLabel.textContent = "Name"; |
| + nameLabel.htmlFor = N.cls("name"); |
| + const name = document.createElement("input"); |
| + name.id = N.cls("name"); |
| + name.maxLength = 64; |
| + name.placeholder = "GitHub deploys"; |
| + name.required = true; |
| + |
| + const nsLabel = document.createElement("label"); |
| + nsLabel.textContent = "Event namespace (events arrive as webhooks.<namespace>)"; |
| + nsLabel.htmlFor = N.cls("ns"); |
| + const ns = document.createElement("input"); |
| + ns.id = N.cls("ns"); |
| + ns.maxLength = 32; |
| + ns.placeholder = "github"; |
| + ns.required = true; |
| + |
| + const targetLabel = document.createElement("label"); |
| + targetLabel.textContent = "Post into"; |
| + targetLabel.htmlFor = N.cls("target"); |
| + const target = document.createElement("select"); |
| + target.id = N.cls("target"); |
| + const spaceOpt = document.createElement("option"); |
| + spaceOpt.value = "space"; |
| + spaceOpt.textContent = "The whole space (space thread)"; |
| + target.appendChild(spaceOpt); |
| + bridge.hooksTargets().then((targets) => { |
| + for (const t of targets) { |
| + const o = document.createElement("option"); |
| + o.value = t.id; |
| + o.textContent = t.name; |
| + target.appendChild(o); |
| + } |
| + }); |
| + |
| + const submit = document.createElement("button"); |
| + submit.type = "submit"; |
| + submit.textContent = "Create webhook"; |
| + const feedback = document.createElement("p"); |
| + feedback.className = N.cls("note"); |
| + feedback.hidden = true; |
| + |
| + form.append(nameLabel, name, nsLabel, ns, targetLabel, target, submit, feedback); |
| + |
| + form.onsubmit = async (e) => { |
| + e.preventDefault(); |
| + const nsVal = ns.value.trim().toLowerCase(); |
| + if (!NS_RE.test(nsVal)) { |
| + show(feedback, "The namespace must be a short lowercase slug — letters, digits, dashes (e.g. github).", true); |
| + return; |
| + } |
| + submit.disabled = true; |
| + show(feedback, "Creating…", false); |
| + const r = await bridge.hooksCreate({ |
| + name: name.value.trim(), |
| + ns: nsVal, |
| + threadId: target.value === "space" ? null : target.value, |
| + space: target.value === "space", |
| + }); |
| + if (r && r.ok) { |
| + paint(el, ctx); // the fresh list shows the new webhook + its URL |
| + } else { |
| + submit.disabled = false; |
| + show(feedback, (r && r.error) || "Couldn't create the webhook — try again.", true); |
| + } |
| + }; |
| + return form; |
| +} |
| + |
| +function show(elm, text, isErr) { |
| + elm.hidden = false; |
| + elm.textContent = text; |
| + elm.className = N.cls("note") + " " + (isErr ? N.cls("note-err") : N.cls("note-ok")); |
| +} |
diff --git a/web/static/newchat.js b/web/static/newchat.js| index 8de0606..143c473 100644 |
| --- a/web/static/newchat.js |
| +++ b/web/static/newchat.js |
| @@ -109,10 +109,17 @@ export function showNewChat(startMode) { |
| ); |
| // The Admin app is system-scope (deliberately absent from personalApps), but |
| // admins get its row in the Apps section all the same — the discoverable way |
| - // in, beyond the /admin command. Null for everyone else. Deliberately exempt |
| - // from the apps gate: Admin is how an admin switches apps back on. |
| - const adminApp = verbs.isAdmin() ? { name: "admin", title: "Admin", glyph: "🛠️", logo: ICON_WRENCH } : null; |
| - const adminMatches = (q) => !!adminApp && adminApp.title.toLowerCase().includes(q); |
| + // in, beyond the /admin command. Empty for everyone else. Deliberately exempt |
| + // from the apps gate: Admin is how an admin switches apps back on. Webhooks |
| + // (also system-scope, also admin-only — docs/webhooks.md) rides the same row |
| + // logic. |
| + const adminApps = verbs.isAdmin() |
| + ? [ |
| + { name: "admin", title: "Admin", glyph: "🛠️", logo: ICON_WRENCH }, |
| + { name: "webhooks", title: "Webhooks", glyph: "📥" }, |
| + ] |
| + : []; |
| + const adminMatching = (q) => adminApps.filter((a) => a.title.toLowerCase().includes(q)); |
| const addAppRow = (listEl, app) => { |
| const b = document.createElement("button"); |
| b.className = "list-row"; |
| @@ -261,12 +268,12 @@ const disc = document.createElement("button"); |
| card.appendChild(p); |
| } |
| const apps = matchingApps(""); |
| - if (apps.length || adminApp) { |
| + if (apps.length || adminApps.length) { |
| const list = document.createElement("div"); |
| list.className = "pick-list"; |
| list.appendChild(pickerSection("Apps")); |
| for (const app of apps) addAppRow(list, app); |
| - if (adminApp) addAppRow(list, adminApp); |
| + for (const app of adminApps) addAppRow(list, app); |
| card.appendChild(list); |
| } |
| return; |
| @@ -444,8 +451,8 @@ const disc = document.createElement("button"); |
| // sidebar; only starting new ones hides. |
| const helpers = ENABLE_BOTS ? others.filter((e) => e.is_agent && rosterMatches(e, q)) : []; |
| const apps = mode === "people" ? matchingApps(q) : []; |
| - const showAdmin = mode === "people" && adminMatches(q); |
| - if (!people.length && !helpers.length && !apps.length && !showAdmin) { |
| + const adminHits = mode === "people" ? adminMatching(q) : []; |
| + if (!people.length && !helpers.length && !apps.length && !adminHits.length) { |
| const p = document.createElement("p"); |
| p.className = "sub"; |
| p.style.padding = "12px 28px"; |
| @@ -453,13 +460,13 @@ const disc = document.createElement("button"); |
| list.appendChild(p); |
| return; |
| } |
| - if (people.length && (helpers.length || apps.length || showAdmin)) list.appendChild(pickerSection("People")); |
| + if (people.length && (helpers.length || apps.length || adminHits.length)) list.appendChild(pickerSection("People")); |
| for (const e of people) addRow(e); |
| if (helpers.length) list.appendChild(pickerSection("Helpers")); |
| for (const e of helpers) addRow(e); |
| - if (apps.length || showAdmin) list.appendChild(pickerSection("Apps")); |
| + if (apps.length || adminHits.length) list.appendChild(pickerSection("Apps")); |
| for (const app of apps) addAppRow(list, app); |
| - if (showAdmin) addAppRow(list, adminApp); |
| + for (const app of adminHits) addAppRow(list, app); |
| }; |
| filter.addEventListener("input", renderList); |
| renderList(); |
diff --git a/web/static/shell.js b/web/static/shell.js| index 3bff5d7..2948f03 100644 |
| --- a/web/static/shell.js |
| +++ b/web/static/shell.js |
| @@ -137,6 +137,7 @@ import { startSetup } from "/static/apps/setup.js"; // the first-run setup conve |
| import { startWelcome } from "/static/apps/welcome.js"; // the joiner welcome conversation (also registers its app) |
| import { planQuickForm } from "/static/quickform.js"; // "Fill in a form instead" — values → verbs+records |
| import "/static/apps/admin.js"; // the admin panel (admin-only; opened via /admin or setup's chip) |
| +import "/static/apps/webhooks.js"; // incoming-webhooks management (admin-only; /webhooks — docs/webhooks.md) |
| import "/static/apps/appstudio.js"; // make + manage your own member apps (docs/member-apps.md) |
| import { openMemoryPanel } from "/static/apps/memory.js"; // /memory — the AI-memory scratchpad (docs/eleven-memory.md) |
| import { setHost as setChatAppHost, setConnected as setChatAppConnected, appendNotice, setNoticeTyping, feedGlideToBottom, scrollToMessage, repaintMentionsOf } from "/static/apps/chat.js"; |
| @@ -5263,6 +5264,19 @@ setElevenTour(async () => { |
| // capabilities it drives. Admin writes authenticate with the member's own device |
| // token — the server accepts it because their user holds the admin role. |
| const adminAuth = () => ({ Authorization: "Bearer " + session.token, "Content-Type": "application/json" }); |
| +// hookIngestURL: where a webhook's requests land — the in.<host> ingest origin |
| +// (docs/webhooks.md), derived from the page host so custom domains follow. |
| +const hookIngestURL = (secret) => `https://in.${location.host}/h/${secret}`; |
| +// fetchHooksService: the hooks sidecar's service identity ({user_id, public_key}, |
| +// {} when none is provisioned) — the seal target for webhook definitions. |
| +async function fetchHooksService() { |
| + try { |
| + const r = await fetch(`/api/hooks/service?token=${encodeURIComponent(session.token)}`); |
| + return r.ok ? await r.json() : null; |
| + } catch { |
| + return null; |
| + } |
| +} |
| // setGroup clears any field it isn't sent, so writes always carry the full |
| // current trio (name + admin name + contact email) with one field changed. |
| // Returns "ok" | "invalid" (the server rejected the value) | "error" (auth / |
| @@ -5299,6 +5313,101 @@ setSetupBridge({ |
| if (me) paintAvatar(me, session.name, () => avatarFor(myUserId)); |
| await publishMyLabel(); |
| }, |
| + // ---- Incoming webhooks (#610, docs/webhooks.md) ---- |
| + // The Webhooks app is pure UI; everything key-shaped happens here. Creation |
| + // is the whole client half of the design: mint the webhook's keypair, wrap |
| + // the target thread's key to it, register it, then seal its private key + |
| + // token + URL secret to the SIDECAR's service identity — the server stores |
| + // only ciphertext, and the URL is known (and shown) instantly. |
| + hooksService: fetchHooksService, |
| + // The threads a webhook can feed: groups I'm in whose key I hold (I must |
| + // wrap it), with the space thread offered separately by the app. |
| + hooksTargets: async () => |
| + threads |
| + .filter((t) => t.kind === "group" && threadKeyB64[t.id]) |
| + .map((t) => ({ id: t.id, name: threadName[t.id] || "Unnamed group" })), |
| + hooksCreate: async ({ name, ns, threadId, space }) => { |
| + try { |
| + const svc = await fetchHooksService(); |
| + if (!svc || !svc.public_key) return { error: "Webhooks aren't set up on this space yet." }; |
| + if (!instanceKey) return { error: "Your keys are still syncing — try again in a moment." }; |
| + let tid = threadId; |
| + if (space) { |
| + tid = await resolveSpaceThread(); |
| + if (!tid) return { error: "The space thread isn't ready yet — try again in a moment." }; |
| + } |
| + const keyB64 = threadKeyB64[tid]; |
| + if (!keyB64) return { error: "You need that chat's key loaded to point a webhook at it — open the chat once first." }; |
| + const kp = await generateUserKeypair(); |
| + const wrapped = await wrapThreadKey(keyB64, kp.publicKey); |
| + const r = await fetch("/api/hooks", { |
| + method: "POST", |
| + headers: adminAuth(), |
| + body: JSON.stringify({ name, public_key: kp.publicKey, thread_id: tid, wrapped }), |
| + }); |
| + if (!r.ok) return { error: `The server refused the webhook (status ${r.status}).` }; |
| + const created = await r.json(); |
| + const secret = randomSecret(); |
| + // Field names are wire format with the Go sidecar (internal/hooks |
| + // Definition) — change them only together. |
| + const def_blob = await sealForPub(svc.public_key, { |
| + v: 1, |
| + id: created.id, |
| + user_id: created.user_id, |
| + name, |
| + ns, |
| + thread_id: tid, |
| + secret, |
| + token: created.token, |
| + priv: kp.privateKey, |
| + }); |
| + const meta_blob = await encrypt(instanceKey, { v: 1, name, ns, thread_id: tid, secret, space: !!space }); |
| + const r2 = await fetch(`/api/hooks/${encodeURIComponent(created.id)}/blobs`, { |
| + method: "PUT", |
| + headers: adminAuth(), |
| + body: JSON.stringify({ def_blob, meta_blob }), |
| + }); |
| + if (!r2.ok) return { error: "Created, but storing its definition failed — remove it and try again." }; |
| + return { ok: true, url: hookIngestURL(secret) }; |
| + } catch (e) { |
| + console.error("webhook create failed:", e); |
| + return { error: "Couldn't create the webhook." }; |
| + } |
| + }, |
| + hooksList: async () => { |
| + try { |
| + const r = await fetch("/api/hooks", { headers: adminAuth() }); |
| + if (!r.ok) return null; |
| + const { hooks } = await r.json(); |
| + const out = []; |
| + for (const h of hooks || []) { |
| + const row = { id: h.id, deployed: (h.deployed_at || 0) > 0 }; |
| + if (h.meta_blob && instanceKey) { |
| + try { |
| + const meta = await decrypt(instanceKey, h.meta_blob); |
| + row.name = meta.name; |
| + row.ns = meta.ns; |
| + row.url = hookIngestURL(meta.secret); |
| + row.target = meta.space ? "the whole space" : threadName[meta.thread_id] || "a group chat"; |
| + } catch { |
| + row.name = "(unreadable webhook)"; |
| + } |
| + } |
| + out.push(row); |
| + } |
| + return out; |
| + } catch { |
| + return null; |
| + } |
| + }, |
| + hooksRemove: async (id) => { |
| + try { |
| + const r = await fetch(`/api/hooks/${encodeURIComponent(id)}`, { method: "DELETE", headers: adminAuth() }); |
| + return r.ok; |
| + } catch { |
| + return false; |
| + } |
| + }, |
| // ---- Moderation (issue #578): report queue + audit log ---- |
| // The Admin app is pure UI; everything key- or name-shaped happens here. |
| // Each queue row's content blob is sealed to THIS admin's key — openSealed |