server: `lchat glue run` — the public-web fetch glue (unfurl + signed image proxy) (#1241)
* server: `lchat glue run` — the public-web fetch glue (unfurl + signed image proxy) The first machine out of the factory (docs/glue.md): a shared, identity-free sibling service that fronts public web content for clients. Two endpoints: - GET /api/unfurl — the instance handler's anonymous twin (no token; the glue has no members to know about). Same URL validation, Accept-Language handling, sealed convergent cache (service-wide salt), stale-serve + background refresh, and mid-fetch waiter coalescing — against the glue's own sqlite store. Card image URLs are rewritten at response time to the signed /img proxy. - GET /img — camo-shaped: serves only URLs this glue itself HMAC-signed while building a card, so it can't be driven as an open proxy, with no identity needed. SSRF-guarded fetch, image/* only, 8MB cap. Salt + signing key mint themselves into the DB on first boot (no env secrets). No request logging by design. Per-IP token buckets (which behind the router collapse to global ones — no client IPs are forwarded, on purpose; hashcash is the planned gate before the client flip makes this load-bearing). Package-main subcommand like links — nothing to extract, it reuses unfurl.go's fetchOG/SSRF guard/seal/previewFlight directly. Not deployed, not yet called by any client: the client flip (config + CSP + host.unfurl), resize, and Giphy-with-our-key are the next slices (docs/glue.md roadmap). No changelog entry: nothing member-visible until the flip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * server+web: env-gated glue client flip + lchat-glue unit - LCHAT_GLUE_URL on an instance names the glue in the chat page's connect-src and injects window.__glueURL; shell.js host.unfurl prefers the glue (no token rides along) and falls back to the instance's own /api/unfurl on transport failure. Unset (the default, and the self-host default on purpose) the page is byte-identical to today's. - deploy/systemd/lchat-glue.service, written by bootstrap everywhere and inert until an operator enables + add-service fronts it (tunnel pattern). Scope: web only for now — native parity (LChatAPI unfurl via the glue) follows before any fleet default-on; canaries exercise the web path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * status: ssh ledger — why-entry for the glue dev stand-up (2026-07-21) ---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
11 files changed,
+819
−1
CLAUDE.md+7 −0bootstrap.go+8 −0deploy/systemd/lchat-glue.service+43 −0docs/glue.md+88 −0glue.go+403 −0glue_test.go+229 −0main.go+18 −1subcommands.go+1 −0web/pages/index.html+3 −0web/static/shell.js+12 −0web/status/ssh/2026-07-21-glue-dev-standup.json+7 −0
diff --git a/CLAUDE.md b/CLAUDE.md| index 134ef75..93c5186 100644 |
| --- a/CLAUDE.md |
| +++ b/CLAUDE.md |
| @@ -165,6 +165,13 @@ in, so deploy is one file (no cgo). |
| Stood up per instance by `lchat add-hooks`. Design + trust model: |
| `docs/webhooks.md`; management UI `web/static/apps/webhooks.js` (admin-only). |
| - `unfurl.go` — OpenGraph link previews (SSRF-guarded fetch + cache). |
| +- `glue.go` — `lchat glue run`, the **public-web fetch glue**: a shared, |
| + identity-free service (unfurl + camo-style HMAC-signed image proxy) that |
| + fronts public web content for clients, so eventually the *instance* never |
| + sees URLs and card-image hosts never see member IPs. Reuses unfurl.go's |
| + fetch/SSRF/sealed-cache machinery; sees URLs attached to nothing; member |
| + content categorically out of scope. Design + trust model + roadmap |
| + (client flip, resize, Giphy-with-our-key): `docs/glue.md`. |
| - `linkscmd.go` — `lchat links run`, the `11ch.at` short-link redirector for |
| message permalinks (issue #905): a bare apex request bounces to |
| `elevenmessenger.com`; `/l/<host>` live-checks the target (SSRF-guarded fetch |
diff --git a/bootstrap.go b/bootstrap.go| index 4615e69..49b53c4 100644 |
| --- a/bootstrap.go |
| +++ b/bootstrap.go |
| @@ -78,6 +78,13 @@ var unitCloud string |
| //go:embed deploy/systemd/lchat-tunnel.service |
| var unitTunnel string |
| +// The public-web fetch glue (`lchat glue run` — see docs/glue.md). Installs |
| +// everywhere, stays inert until an operator enables it and fronts it with |
| +// `add-service`; instances opt in by pointing LCHAT_GLUE_URL at that route. |
| +// |
| +//go:embed deploy/systemd/lchat-glue.service |
| +var unitGlue string |
| + |
| // The auto-updater (updater.go): a root oneshot fired hourly by its timer. |
| // Enabled by bootstrap — a pass with no reachable release feed is a no-op, so |
| // it's safe everywhere, and a box provisioned tomorrow auto-updates without a |
| @@ -496,6 +503,7 @@ func runBootstrap(args []string) { |
| write(filepath.Join(*unitDir, "lchat-control.service"), unitControl) |
| write(filepath.Join(*unitDir, "lchat-cloud.service"), unitCloud) |
| write(filepath.Join(*unitDir, "lchat-tunnel.service"), unitTunnel) |
| + write(filepath.Join(*unitDir, "lchat-glue.service"), unitGlue) |
| write(filepath.Join(*unitDir, "lchat-update.service"), unitUpdate) |
| write(filepath.Join(*unitDir, "lchat-update.timer"), unitUpdateTimer) |
| write("/etc/tmpfiles.d/lchat.conf", tmpfilesConf) |
diff --git a/deploy/systemd/lchat-glue.service b/deploy/systemd/lchat-glue.service| new file mode 100644 |
| index 0000000..7f066a1 |
| --- /dev/null |
| +++ b/deploy/systemd/lchat-glue.service |
| @@ -0,0 +1,43 @@ |
| +[Unit] |
| +Description=lchat public-web fetch glue (unfurl + signed image proxy) |
| +After=network-online.target |
| +Wants=network-online.target |
| + |
| +[Service] |
| +# No env file: the glue's secrets (preview salt + image-signing key) mint |
| +# themselves into its DB on first boot (glue.go); nothing else to configure. |
| +ExecStart=/opt/lchat/lchat glue run --socket /run/lchat/glue.sock --db /data/glue.db |
| +Restart=always |
| +RestartSec=1 |
| + |
| +# --- hardening (see docs/instance-router.md § Hardening) --- |
| +User=lchat |
| +Group=lchat |
| +CapabilityBoundingSet= |
| +NoNewPrivileges=yes |
| +ProtectSystem=strict |
| +ProtectHome=yes |
| +ReadWritePaths=/run/lchat /data |
| +PrivateTmp=yes |
| +PrivateDevices=yes |
| +ProtectClock=yes |
| +ProtectHostname=yes |
| +ProtectKernelTunables=yes |
| +ProtectKernelModules=yes |
| +ProtectKernelLogs=yes |
| +ProtectControlGroups=yes |
| +ProtectProc=invisible |
| +RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX |
| +RestrictNamespaces=yes |
| +RestrictRealtime=yes |
| +RestrictSUIDSGID=yes |
| +LockPersonality=yes |
| +MemoryDenyWriteExecute=yes |
| +SystemCallFilter=@system-service |
| +SystemCallErrorNumber=EPERM |
| +SystemCallArchitectures=native |
| +MemoryMax=512M |
| +TasksMax=256 |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
diff --git a/docs/glue.md b/docs/glue.md| new file mode 100644 |
| index 0000000..d250a34 |
| --- /dev/null |
| +++ b/docs/glue.md |
| @@ -0,0 +1,88 @@ |
| +# 🤖 The public-web fetch glue (`lchat glue run`) |
| + |
| +A shared, identity-free service that fronts **public web content** for |
| +clients: OpenGraph unfurls and a camo-style signed image proxy for the cards' |
| +images. One binary subcommand (`glue.go`), one sqlite file, fronted by the |
| +instance router like links/push/home. Born in the unfurl-through-Finland |
| +discussion (PRs #1237/#1240 → this): if a service is going to see posted |
| +URLs at all, it should be a service that knows *nothing else*. |
| + |
| +## Why it exists |
| + |
| +Unfurling is the messenger's one documented E2E exception: to build a link |
| +card, *some* server must see the URL. Today that server is the member's own |
| +instance. Moving the fetch to shared glue buys, in rough order of value: |
| + |
| +1. **The instance can stop seeing URLs entirely** (once clients call the glue |
| + directly — the phase-2 flip below). The instance server then stores and |
| + relays only ciphertext, full stop; the unfurl exception moves to a service |
| + that holds no members, threads, tokens, or instance list. |
| +2. **Viewers stop leaking their IPs to card-image hosts.** The web client |
| + sets `img.src` to the raw `og:image` URL today, so every *viewer* of a |
| + card fetches from a third-party host; `http:`-only images die silently as |
| + mixed content. The glue's `/img` proxy fixes both. |
| +3. **Self-hosters stop exposing their egress IP.** A member can post a link |
| + to a server they control and learn the instance's IP — for a home |
| + self-hoster, their home IP. Pointing unfurl at the glue hides it; running |
| + their own glue keeps full autonomy. |
| +4. **A shared warm cache.** Popular links unfurl once per language across |
| + every space that uses the glue, and the poster/recipient in-flight window |
| + mostly disappears because the cache is usually already warm. |
| + |
| +## Trust model |
| + |
| +- **Sees:** posted URLs + an `Accept-Language` tag, transiently, attached to |
| + nothing — no token, no instance id, no member id. Behind the router it |
| + doesn't even see client IPs (nothing forwards them, deliberately). |
| +- **Stores:** the sealed preview cache — same convergent-encryption scheme as |
| + the instance's (`unfurl.go`): row key = salted hash of language+URL, blob |
| + opens only with a key derived from language+URL, service-wide salt (minted |
| + on first boot, lives only in the glue's DB) defeats precomputed |
| + dictionaries. The cache at rest reveals nothing a reader doesn't already |
| + know the URL for. No request logs; the only log line is startup. |
| +- **Never:** member content. Member photos/files are encrypted client-side |
| + (`processImage` + `encryptBytes`) and are categorically out of scope — the |
| + glue's remit is content already on the public web. Any future endpoint that |
| + would touch member bytes is a design error; say no. |
| +- **Not an open proxy:** `/img` serves only URLs the glue itself HMAC-signed |
| + while building a card (GitHub-camo shape), so third parties can't drive it |
| + as a generic fetcher. Signing is at response time; the cache stores raw |
| + URLs, so the key could rotate without invalidating cached cards. |
| +- **SSRF:** every outbound dial (unfurl page + image, every redirect hop) |
| + re-validates the resolved IP via `guardDialControl` — same guard as the |
| + instance, links, and push. |
| + |
| +## Abuse posture (v1: honest, modest) |
| + |
| +`/api/unfurl` and `/img` carry per-IP token buckets, which behind the router |
| +collapse into global buckets (no forwarded IPs — a privacy feature with a |
| +rate-limiting cost). That is acceptable while nothing points at the glue by |
| +default. Before the client flip makes it load-bearing, add **hashcash** (the |
| +cloud provisioner already ships the machinery) so anonymous callers pay in |
| +CPU instead of identity. |
| + |
| +## Deployment |
| + |
| +Standard sibling-service shape: `lchat glue run` behind the router on a unix |
| +socket (`-socket /run/lchat/glue.sock`), DB at `/data/glue.db` |
| +(`LCHAT_GLUE_DB`), service route + `/etc/hosts` pin like cloud/home. Not |
| +deployed anywhere yet; when it ships, the planet-scale guide gains its row. |
| +No secrets in env — the salt and signing key mint themselves into the DB on |
| +first boot. |
| + |
| +## Roadmap |
| + |
| +- **Phase 2 — the client flip.** Instance serves the glue URL to clients |
| + (config), CSP grows `connect-src`/`img-src` for it, `host.unfurl` prefers |
| + the glue with instance-unfurl fallback, and hashcash lands with it. |
| + Self-host default stays *local fetch* — a self-hosted instance silently |
| + shipping its members' URLs to our glue would betray the self-host story; |
| + pointing at shared glue (or their own) is opt-in config. |
| +- **Resize.** Public card images only (og:image can be multi-MB); a flag on |
| + the `/img` path, never a new trust decision. Member photos keep their |
| + client-side pipeline. |
| +- **Giphy.** Same service family, authenticated upstream: search rides the |
| + glue with *our* API key (kills the get-an-API-key wall for spaces), the |
| + space-supplied key remains as override (docs/app-credentials.md). To be |
| + honest about the trade: today Giphy search is client→Giphy (we see |
| + nothing); via glue we see search terms, identity-free. |
diff --git a/glue.go b/glue.go| new file mode 100644 |
| index 0000000..b0c731b |
| --- /dev/null |
| +++ b/glue.go |
| @@ -0,0 +1,403 @@ |
| +// `lchat glue run` — the public-web fetch glue (docs/glue.md). A shared, |
| +// identity-free service that fronts *public web content* for clients: OpenGraph |
| +// unfurls and an HMAC-signed image proxy for the cards' images. Like links |
| +// (linkscmd.go) it's a subcommand of this binary, not a separate module — |
| +// it deliberately reuses the instance's own unfurl machinery (unfurl.go: |
| +// fetchOG, the SSRF dial guard, the sealed convergent cache, previewFlight). |
| +// |
| +// Trust model, in one breath: the glue sees URLs (the documented unfurl |
| +// exception) attached to NOTHING — no token, no instance, no member. Behind |
| +// the router it doesn't even see client IPs (nothing forwards them — that's a |
| +// feature, with the honest cost that per-IP rate buckets collapse to global |
| +// ones there; hashcash is the planned real gate, see docs/glue.md). It must |
| +// never grow an endpoint that touches member content: member photos/files are |
| +// encrypted client-side and are categorically out of scope here. |
| +// |
| +// The image proxy is camo-shaped (GitHub's trick): /img serves only URLs the |
| +// unfurler itself signed when it built a card, so it can't be driven as an |
| +// open proxy, with no identity needed. Signing happens at response time (the |
| +// cache stores the raw og:image URL), so the key could rotate without |
| +// invalidating cached cards. |
| +package main |
| + |
| +import ( |
| + "crypto/hmac" |
| + "crypto/rand" |
| + "crypto/sha256" |
| + "database/sql" |
| + "encoding/base64" |
| + "encoding/hex" |
| + "flag" |
| + "fmt" |
| + "io" |
| + "log" |
| + "net" |
| + "net/http" |
| + "net/url" |
| + "os" |
| + "strings" |
| + "sync" |
| + "time" |
| +) |
| + |
| +const ( |
| + glueImgMaxBytes = 8 * 1024 * 1024 // largest card image we'll proxy |
| + glueImgTimeout = 15 * time.Second |
| + glueImgCache = "public, max-age=86400" |
| +) |
| + |
| +// glueImgClient fetches card images. Same SSRF posture as ogClient — every |
| +// dial (and every redirect hop) re-validates the resolved IP. |
| +var glueImgClient = &http.Client{ |
| + Timeout: glueImgTimeout, |
| + Transport: &http.Transport{ |
| + DialContext: (&net.Dialer{ |
| + Timeout: glueImgTimeout, |
| + Control: guardDialControl, |
| + }).DialContext, |
| + TLSHandshakeTimeout: glueImgTimeout, |
| + ResponseHeaderTimeout: glueImgTimeout, |
| + DisableKeepAlives: true, |
| + }, |
| + CheckRedirect: func(req *http.Request, via []*http.Request) error { |
| + if len(via) >= 5 { |
| + return fmt.Errorf("too many redirects") |
| + } |
| + if req.URL.Scheme != "http" && req.URL.Scheme != "https" { |
| + return fmt.Errorf("non-http redirect") |
| + } |
| + return nil |
| + }, |
| +} |
| + |
| +type glueServer struct { |
| + db *sql.DB |
| + salt string // service-wide preview salt (convergent-encryption pepper) |
| + signKey []byte // HMAC key for /img URLs; minted once, lives only in the DB |
| + // inflight coalesces concurrent unfurls exactly like the instance's |
| + // (main.go): one fetch per URL+language, everyone shares the result. |
| + inflight sync.Map |
| + // Ceilings, not budgets — and behind the router every caller shares one |
| + // bucket (no client IPs are forwarded, on purpose). See the package doc. |
| + unfurlLimiter *modLimiter |
| + imgLimiter *modLimiter |
| +} |
| + |
| +func newGlueServer(dbPath string) (*glueServer, error) { |
| + d, err := sql.Open("sqlite", dbPath) |
| + if err != nil { |
| + return nil, err |
| + } |
| + d.SetMaxOpenConns(1) |
| + for _, p := range []string{ |
| + "PRAGMA journal_mode=WAL", |
| + "PRAGMA busy_timeout=5000", |
| + "PRAGMA synchronous=NORMAL", |
| + } { |
| + if _, err := d.Exec(p); err != nil { |
| + return nil, err |
| + } |
| + } |
| + for _, ddl := range []string{ |
| + `CREATE TABLE IF NOT EXISTS config(key TEXT PRIMARY KEY, value TEXT NOT NULL)`, |
| + // Same sealed shape as the instance's link_previews_sealed: the row key |
| + // is a salted hash and the blob opens only with a key derived from the |
| + // language+URL, so the shared cache at rest reveals nothing a reader |
| + // doesn't already know the URL for. |
| + `CREATE TABLE IF NOT EXISTS glue_previews( |
| + url_hash TEXT PRIMARY KEY, |
| + blob TEXT NOT NULL, |
| + fetched_at INTEGER NOT NULL)`, |
| + } { |
| + if _, err := d.Exec(ddl); err != nil { |
| + return nil, err |
| + } |
| + } |
| + g := &glueServer{ |
| + db: d, |
| + unfurlLimiter: newModLimiter(30, 30.0/60), |
| + imgLimiter: newModLimiter(240, 120.0/60), |
| + } |
| + if g.salt, err = glueConfigSecret(d, "preview_salt"); err != nil { |
| + return nil, err |
| + } |
| + keyHex, err := glueConfigSecret(d, "img_sign_key") |
| + if err != nil { |
| + return nil, err |
| + } |
| + if g.signKey, err = hex.DecodeString(keyHex); err != nil { |
| + return nil, err |
| + } |
| + return g, nil |
| +} |
| + |
| +// glueConfigSecret returns the named config value, minting a random one on |
| +// first boot — the same insert-if-absent pattern as the instance's |
| +// previewSalt, so restarts and concurrent starts converge on one value. |
| +func glueConfigSecret(d *sql.DB, key string) (string, error) { |
| + buf := make([]byte, 32) |
| + if _, err := rand.Read(buf); err != nil { |
| + return "", err |
| + } |
| + if _, err := d.Exec( |
| + "INSERT OR IGNORE INTO config(key, value) VALUES(?, ?)", key, hex.EncodeToString(buf)); err != nil { |
| + return "", err |
| + } |
| + var v string |
| + err := d.QueryRow("SELECT value FROM config WHERE key = ?", key).Scan(&v) |
| + return v, err |
| +} |
| + |
| +func (g *glueServer) handler() http.Handler { |
| + mux := http.NewServeMux() |
| + mux.HandleFunc("GET /api/unfurl", g.serveUnfurl) |
| + mux.HandleFunc("GET /img", g.serveImage) |
| + mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) { |
| + fmt.Fprintln(w, "ok") |
| + }) |
| + mux.HandleFunc("GET /api/version", func(w http.ResponseWriter, r *http.Request) { |
| + writeJSON(w, http.StatusOK, map[string]any{"service": "glue", "build": buildVersion}) |
| + }) |
| + // The glue has no pages of its own; a stray visit goes to the real site. |
| + mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { |
| + http.Redirect(w, r, "https://elevenmessenger.com", http.StatusFound) |
| + }) |
| + return mux |
| +} |
| + |
| +// glueCallerKey is the rate-limit bucket key. Mirrors linkscmd.go — behind |
| +// the router (unix socket, no forwarded IPs) this degrades to one shared |
| +// bucket, which is the accepted posture for now (package doc). |
| +func glueCallerKey(r *http.Request) string { |
| + ip, _, err := net.SplitHostPort(r.RemoteAddr) |
| + if err != nil { |
| + return r.RemoteAddr |
| + } |
| + return ip |
| +} |
| + |
| +// serveUnfurl is the instance handler's anonymous twin: no token, same URL |
| +// validation, same language handling, same stale-serve semantics — but the |
| +// card's image URL is rewritten to the signed /img proxy before it leaves. |
| +func (g *glueServer) serveUnfurl(w http.ResponseWriter, r *http.Request) { |
| + if !g.unfurlLimiter.allow(glueCallerKey(r)) { |
| + httpError(w, http.StatusTooManyRequests, "slow down") |
| + return |
| + } |
| + raw := strings.TrimSpace(r.URL.Query().Get("url")) |
| + u, err := url.Parse(raw) |
| + if err != nil || u.Host == "" || (u.Scheme != "http" && u.Scheme != "https") || len(raw) > 2048 { |
| + httpError(w, http.StatusBadRequest, "bad url") |
| + return |
| + } |
| + norm := u.String() |
| + lang := previewLang(r.Header.Get("Accept-Language")) |
| + |
| + if pv, ok := g.getPreview(norm, lang); ok { |
| + writeJSON(w, http.StatusOK, g.signedCard(r, pv)) |
| + ttl := int64(previewFreshTTL) |
| + if !pv.OK { |
| + ttl = previewFailTTL |
| + } |
| + if time.Now().Unix()-pv.FetchedAt > ttl { |
| + go g.fetchAndCache(norm, lang) |
| + } |
| + return |
| + } |
| + writeJSON(w, http.StatusOK, g.signedCard(r, g.fetchAndCache(norm, lang))) |
| +} |
| + |
| +// fetchAndCache mirrors the instance's fetchAndCachePreview (main.go), |
| +// including the mid-fetch waiters fix — against the glue's own store. |
| +func (g *glueServer) fetchAndCache(norm, lang string) LinkPreview { |
| + flightKey := previewCacheKey(lang, norm) |
| + f := &previewFlight{done: make(chan struct{})} |
| + if cur, busy := g.inflight.LoadOrStore(flightKey, f); busy { |
| + flight := cur.(*previewFlight) |
| + <-flight.done |
| + return flight.pv |
| + } |
| + defer func() { |
| + g.inflight.Delete(flightKey) |
| + close(f.done) |
| + }() |
| + |
| + pv, err := fetchOG(norm, lang) |
| + if err != nil { |
| + pv = LinkPreview{URL: norm, OK: false} |
| + } |
| + pv.URL = norm |
| + pv.FetchedAt = time.Now().Unix() |
| + g.putPreview(pv, lang) |
| + f.pv = pv |
| + return pv |
| +} |
| + |
| +func (g *glueServer) getPreview(u, lang string) (LinkPreview, bool) { |
| + key := previewCacheKey(lang, u) |
| + var blob string |
| + var fetchedAt int64 |
| + err := g.db.QueryRow( |
| + "SELECT blob, fetched_at FROM glue_previews WHERE url_hash = ?", previewURLHash(g.salt, key), |
| + ).Scan(&blob, &fetchedAt) |
| + if err != nil { |
| + return LinkPreview{}, false |
| + } |
| + pv, ok := openPreview(g.salt, key, blob) |
| + if !ok { |
| + return LinkPreview{}, false |
| + } |
| + pv.URL = u |
| + pv.FetchedAt = fetchedAt |
| + return pv, true |
| +} |
| + |
| +func (g *glueServer) putPreview(pv LinkPreview, lang string) { |
| + key := previewCacheKey(lang, pv.URL) |
| + blob, err := sealPreview(g.salt, key, pv) |
| + if err != nil { |
| + return |
| + } |
| + // Same put-time TTL sweep as the instance cache: prunes with activity. |
| + g.db.Exec("DELETE FROM glue_previews WHERE fetched_at < ?", time.Now().Unix()-previewMaxAge) |
| + g.db.Exec( |
| + `INSERT INTO glue_previews(url_hash, blob, fetched_at) VALUES(?, ?, ?) |
| + ON CONFLICT(url_hash) DO UPDATE SET blob=excluded.blob, fetched_at=excluded.fetched_at`, |
| + previewURLHash(g.salt, key), blob, pv.FetchedAt) |
| +} |
| + |
| +// signedCard swaps a card's raw og:image URL for the signed /img proxy URL, |
| +// so the viewer's browser talks to the glue, not the image's host (no member |
| +// IP reaches it, and http-only images stop dying as mixed content). |
| +func (g *glueServer) signedCard(r *http.Request, pv LinkPreview) LinkPreview { |
| + if pv.Image != "" { |
| + pv.Image = g.proxyImageURL(r, pv.Image) |
| + } |
| + return pv |
| +} |
| + |
| +func (g *glueServer) proxyImageURL(r *http.Request, img string) string { |
| + mac := hmac.New(sha256.New, g.signKey) |
| + mac.Write([]byte(img)) |
| + scheme := "https" |
| + if r.TLS == nil && glueLoopbackHost(r.Host) { |
| + scheme = "http" // tests and bare local runs; behind the router TLS is the router's |
| + } |
| + return scheme + "://" + r.Host + "/img?u=" + |
| + base64.RawURLEncoding.EncodeToString([]byte(img)) + |
| + "&s=" + hex.EncodeToString(mac.Sum(nil)) |
| +} |
| + |
| +func glueLoopbackHost(hostport string) bool { |
| + host, _, err := net.SplitHostPort(hostport) |
| + if err != nil { |
| + host = hostport |
| + } |
| + if host == "localhost" { |
| + return true |
| + } |
| + ip := net.ParseIP(host) |
| + return ip != nil && ip.IsLoopback() |
| +} |
| + |
| +// serveImage is the camo half: verify the signature (only URLs this glue |
| +// itself put in a card are proxyable — never an open proxy), then fetch and |
| +// re-serve the image over our origin. Buffered rather than streamed so an |
| +// oversize image is rejected cleanly instead of truncated mid-body; the size |
| +// cap bounds the memory. |
| +func (g *glueServer) serveImage(w http.ResponseWriter, r *http.Request) { |
| + if !g.imgLimiter.allow(glueCallerKey(r)) { |
| + httpError(w, http.StatusTooManyRequests, "slow down") |
| + return |
| + } |
| + rawImg, err := base64.RawURLEncoding.DecodeString(r.URL.Query().Get("u")) |
| + if err != nil { |
| + httpError(w, http.StatusBadRequest, "bad url") |
| + return |
| + } |
| + sig, err := hex.DecodeString(r.URL.Query().Get("s")) |
| + if err != nil { |
| + httpError(w, http.StatusBadRequest, "bad signature") |
| + return |
| + } |
| + mac := hmac.New(sha256.New, g.signKey) |
| + mac.Write(rawImg) |
| + if !hmac.Equal(sig, mac.Sum(nil)) { |
| + httpError(w, http.StatusForbidden, "bad signature") |
| + return |
| + } |
| + imgURL, err := url.Parse(string(rawImg)) |
| + if err != nil || (imgURL.Scheme != "http" && imgURL.Scheme != "https") { |
| + httpError(w, http.StatusBadRequest, "bad url") |
| + return |
| + } |
| + |
| + req, err := http.NewRequestWithContext(r.Context(), "GET", imgURL.String(), nil) |
| + if err != nil { |
| + httpError(w, http.StatusBadGateway, "fetch failed") |
| + return |
| + } |
| + req.Header.Set("User-Agent", "lchat-glue/1.0 (+https://elevenmessenger.com)") |
| + req.Header.Set("Accept", "image/*") |
| + resp, err := glueImgClient.Do(req) |
| + if err != nil { |
| + httpError(w, http.StatusBadGateway, "fetch failed") |
| + return |
| + } |
| + defer resp.Body.Close() |
| + ct := strings.ToLower(resp.Header.Get("Content-Type")) |
| + if resp.StatusCode != http.StatusOK || !strings.HasPrefix(ct, "image/") { |
| + httpError(w, http.StatusBadGateway, "not an image") |
| + return |
| + } |
| + body, err := io.ReadAll(io.LimitReader(resp.Body, glueImgMaxBytes+1)) |
| + if err != nil || len(body) > glueImgMaxBytes { |
| + httpError(w, http.StatusBadGateway, "image too large") |
| + return |
| + } |
| + w.Header().Set("Content-Type", resp.Header.Get("Content-Type")) |
| + w.Header().Set("Cache-Control", glueImgCache) |
| + w.Header().Set("X-Content-Type-Options", "nosniff") |
| + w.Write(body) |
| +} |
| + |
| +// runGlueCmd dispatches `lchat glue <command>`, mirroring runLinksCmd. |
| +func runGlueCmd(args []string) { |
| + if len(args) < 1 { |
| + fmt.Fprintln(os.Stderr, `usage: lchat glue <command> |
| + |
| +commands: |
| + run serve the public-web fetch glue (unfurl + signed image proxy; docs/glue.md)`) |
| + os.Exit(2) |
| + } |
| + switch args[0] { |
| + case "run": |
| + runGlueRun(args[1:]) |
| + default: |
| + die("glue: unknown command %q", args[0]) |
| + } |
| +} |
| + |
| +// runGlueRun is `lchat glue run`. |
| +func runGlueRun(args []string) { |
| + fs := flag.NewFlagSet("glue", flag.ExitOnError) |
| + addr := fs.String("addr", envOr("ADDR", ":1749"), "listen address (env: ADDR)") |
| + socket := fs.String("socket", envOr("LCHAT_SOCKET", ""), "unix socket to serve on instead of -addr (env: LCHAT_SOCKET) — how the router fronts this service") |
| + dbPath := fs.String("db", envOr("LCHAT_GLUE_DB", "glue.db"), "sqlite path for the sealed preview cache + service secrets (env: LCHAT_GLUE_DB)") |
| + _ = fs.Parse(args) |
| + |
| + g, err := newGlueServer(*dbPath) |
| + if err != nil { |
| + die("glue: %v", err) |
| + } |
| + ln, addrDesc, err := listen(*addr, *socket) |
| + if err != nil { |
| + die("glue: %v", err) |
| + } |
| + // Deliberately the only startup log line — the glue never logs URLs. |
| + log.Printf("glue: serving on %s", addrDesc) |
| + srv := &http.Server{Handler: g.handler(), ReadHeaderTimeout: 10 * time.Second} |
| + if err := srv.Serve(ln); err != nil { |
| + log.Fatal(err) |
| + } |
| +} |
diff --git a/glue_test.go b/glue_test.go| new file mode 100644 |
| index 0000000..f8724c7 |
| --- /dev/null |
| +++ b/glue_test.go |
| @@ -0,0 +1,229 @@ |
| +package main |
| + |
| +import ( |
| + "encoding/json" |
| + "io" |
| + "net/http" |
| + "net/http/httptest" |
| + "net/url" |
| + "strings" |
| + "testing" |
| +) |
| + |
| +func newTestGlue(t *testing.T) (*httptest.Server, *glueServer) { |
| + t.Helper() |
| + g, err := newGlueServer(t.TempDir() + "/glue.db") |
| + if err != nil { |
| + t.Fatalf("newGlueServer: %v", err) |
| + } |
| + ts := httptest.NewServer(g.handler()) |
| + t.Cleanup(ts.Close) |
| + t.Cleanup(func() { g.db.Close() }) |
| + return ts, g |
| +} |
| + |
| +// The glue unfurls anonymously (no token — it has no members to know about), |
| +// rewrites the card's image to its signed /img proxy, and that proxy URL |
| +// actually serves the image bytes back over the glue's own origin. |
| +func TestGlueUnfurlAndImageProxy(t *testing.T) { |
| + ts, g := newTestGlue(t) |
| + |
| + allowPrivatePreviewHosts = true |
| + t.Cleanup(func() { allowPrivatePreviewHosts = false }) |
| + |
| + imgBytes := []byte("not-really-a-png-but-served-as-one") |
| + site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + if r.URL.Path == "/pic.png" { |
| + w.Header().Set("Content-Type", "image/png") |
| + w.Write(imgBytes) |
| + return |
| + } |
| + w.Header().Set("Content-Type", "text/html") |
| + w.Write([]byte(`<head><meta property="og:title" content="Glued">` + |
| + `<meta property="og:image" content="/pic.png"></head>`)) |
| + })) |
| + t.Cleanup(site.Close) |
| + |
| + r, err := http.Get(ts.URL + "/api/unfurl?url=" + url.QueryEscape(site.URL)) |
| + if err != nil { |
| + t.Fatalf("unfurl: %v", err) |
| + } |
| + var pv LinkPreview |
| + json.NewDecoder(r.Body).Decode(&pv) |
| + r.Body.Close() |
| + if !pv.OK || pv.Title != "Glued" { |
| + t.Fatalf("preview = %+v", pv) |
| + } |
| + // The image is no longer the site's URL — it's the glue's signed proxy. |
| + if !strings.HasPrefix(pv.Image, ts.URL+"/img?") { |
| + t.Fatalf("image not proxied: %q", pv.Image) |
| + } |
| + ir, err := http.Get(pv.Image) |
| + if err != nil { |
| + t.Fatalf("proxied image: %v", err) |
| + } |
| + defer ir.Body.Close() |
| + if ir.StatusCode != http.StatusOK || ir.Header.Get("Content-Type") != "image/png" { |
| + t.Fatalf("proxied image: status %d, type %q", ir.StatusCode, ir.Header.Get("Content-Type")) |
| + } |
| + got, err := io.ReadAll(ir.Body) |
| + if err != nil || string(got) != string(imgBytes) { |
| + t.Fatalf("proxied bytes = %q err=%v", got, err) |
| + } |
| + |
| + // Cached: a second unfurl works with the cache row (and stays sealed at |
| + // rest — no plaintext in the stored row). |
| + var urlHash, blob string |
| + if err := g.db.QueryRow("SELECT url_hash, blob FROM glue_previews").Scan(&urlHash, &blob); err != nil { |
| + t.Fatalf("raw row: %v", err) |
| + } |
| + for _, secret := range []string{site.URL, "Glued", "pic.png"} { |
| + if strings.Contains(urlHash+blob, secret) { |
| + t.Fatalf("plaintext %q found in the stored row", secret) |
| + } |
| + } |
| +} |
| + |
| +// /img is not an open proxy: an unsigned or tampered URL never fetches. |
| +func TestGlueImageProxyRejectsUnsigned(t *testing.T) { |
| + ts, g := newTestGlue(t) |
| + |
| + // A URL the glue never signed: correct encoding, wrong signature. |
| + u := "aHR0cDovL2V4YW1wbGUuY29tL3guanBn" // base64url("http://example.com/x.jpg") |
| + for _, q := range []string{ |
| + "u=" + u + "&s=" + strings.Repeat("00", 32), |
| + "u=" + u, |
| + "u=%%%&s=zz", |
| + } { |
| + r, err := http.Get(ts.URL + "/img?" + q) |
| + if err != nil { |
| + t.Fatalf("img %q: %v", q, err) |
| + } |
| + r.Body.Close() |
| + if r.StatusCode == http.StatusOK { |
| + t.Fatalf("img %q: served without a valid signature", q) |
| + } |
| + } |
| + |
| + // And a signed URL to a non-image is refused after the fetch. |
| + allowPrivatePreviewHosts = true |
| + t.Cleanup(func() { allowPrivatePreviewHosts = false }) |
| + site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + w.Header().Set("Content-Type", "text/html") |
| + w.Write([]byte("<html>not an image</html>")) |
| + })) |
| + t.Cleanup(site.Close) |
| + req, _ := http.NewRequest("GET", ts.URL+"/api/unfurl", nil) |
| + signed := g.proxyImageURL(&http.Request{Host: strings.TrimPrefix(ts.URL, "http://"), URL: req.URL}, site.URL) |
| + r, err := http.Get(signed) |
| + if err != nil { |
| + t.Fatalf("signed non-image: %v", err) |
| + } |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusBadGateway { |
| + t.Fatalf("signed non-image: want 502, got %d", r.StatusCode) |
| + } |
| +} |
| + |
| +// Language variants cache as separate rows on the glue too — same contract |
| +// as the instance (TestUnfurlLanguage), shared-cache edition. |
| +func TestGlueUnfurlLanguageVariants(t *testing.T) { |
| + ts, g := newTestGlue(t) |
| + |
| + allowPrivatePreviewHosts = true |
| + t.Cleanup(func() { allowPrivatePreviewHosts = false }) |
| + |
| + site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + w.Header().Set("Content-Type", "text/html") |
| + w.Write([]byte(`<head><meta property="og:title" content="lang:` + |
| + r.Header.Get("Accept-Language") + `"></head>`)) |
| + })) |
| + t.Cleanup(site.Close) |
| + |
| + fetch := func(lang string) string { |
| + t.Helper() |
| + req, _ := http.NewRequest("GET", ts.URL+"/api/unfurl?url="+url.QueryEscape(site.URL), nil) |
| + if lang != "" { |
| + req.Header.Set("Accept-Language", lang) |
| + } |
| + r, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatalf("unfurl(%q): %v", lang, err) |
| + } |
| + defer r.Body.Close() |
| + var pv LinkPreview |
| + json.NewDecoder(r.Body).Decode(&pv) |
| + return pv.Title |
| + } |
| + |
| + if got := fetch("fi"); got != "lang:fi" { |
| + t.Fatalf("fi = %q", got) |
| + } |
| + if got := fetch("en-IE,en;q=0.9"); got != "lang:en-IE, en;q=0.8" { |
| + t.Fatalf("en-IE = %q", got) |
| + } |
| + var rows int |
| + g.db.QueryRow("SELECT COUNT(*) FROM glue_previews").Scan(&rows) |
| + if rows != 2 { |
| + t.Fatalf("cache rows = %d, want 2", rows) |
| + } |
| +} |
| + |
| +// The client flip is env-gated: with LCHAT_GLUE_URL set, the chat page names |
| +// the glue in connect-src and hands the origin to shell.js; unset, the page |
| +// is byte-identical to today's (self-host default = local unfurl). |
| +func TestGlueClientFlipInjection(t *testing.T) { |
| + ts, srv := newTestServer(t) |
| + |
| + get := func() (csp, body string) { |
| + t.Helper() |
| + r, err := http.Get(ts.URL + "/") |
| + if err != nil { |
| + t.Fatalf("GET /: %v", err) |
| + } |
| + defer r.Body.Close() |
| + b, _ := io.ReadAll(r.Body) |
| + return r.Header.Get("Content-Security-Policy"), string(b) |
| + } |
| + |
| + csp, body := get() |
| + if strings.Contains(csp, "glue") { |
| + t.Fatalf("CSP names a glue with none configured: %s", csp) |
| + } |
| + if !strings.Contains(body, `window.__glueURL = ""`) { |
| + t.Fatal("page should inject an empty glue URL when unconfigured") |
| + } |
| + |
| + srv.glueURL = "https://glue.example" |
| + csp, body = get() |
| + if !strings.Contains(csp, "connect-src 'self' https://*.giphy.com") || |
| + !strings.Contains(csp, " https://glue.example; ") { |
| + t.Fatalf("CSP missing the glue origin: %s", csp) |
| + } |
| + if !strings.Contains(body, `window.__glueURL = "https://glue.example"`) { |
| + t.Fatal("page missing the injected glue URL") |
| + } |
| +} |
| + |
| +// Service secrets survive a restart: the same DB yields the same salt and |
| +// signing key, so cached rows still open and signed URLs stay valid. |
| +func TestGlueSecretsPersist(t *testing.T) { |
| + dir := t.TempDir() |
| + g1, err := newGlueServer(dir + "/glue.db") |
| + if err != nil { |
| + t.Fatalf("first open: %v", err) |
| + } |
| + salt, key := g1.salt, g1.signKey |
| + g1.db.Close() |
| + g2, err := newGlueServer(dir + "/glue.db") |
| + if err != nil { |
| + t.Fatalf("reopen: %v", err) |
| + } |
| + defer g2.db.Close() |
| + if g2.salt != salt || string(g2.signKey) != string(key) { |
| + t.Fatal("glue secrets changed across restart") |
| + } |
| + if len(key) != 32 || salt == "" { |
| + t.Fatalf("weak secrets: key %d bytes, salt %q", len(key), salt) |
| + } |
| +} |
diff --git a/main.go b/main.go| index 93820c8..72056ee 100644 |
| --- a/main.go |
| +++ b/main.go |
| @@ -83,6 +83,11 @@ type Server struct { |
| // <homeOrigin>/enroll as a user-initiated popup to back this instance up (see |
| // the enrolment cross-origin contract); set LCHAT_HOME_ORIGIN per deployment. |
| homeOrigin string |
| + // The public-web fetch glue this instance's clients should prefer for |
| + // unfurls (docs/glue.md) — an origin like https://glue.example, set via |
| + // LCHAT_GLUE_URL. Empty (the default, and the self-host default on |
| + // purpose) keeps unfurling on this instance's own /api/unfurl. |
| + glueURL string |
| // The cloud ledger's plan flag, mirrored into this instance's env by the |
| // provisioner (LCHAT_ENTITLEMENT, "free"/"paid"/"demo"). Gates the space |
| // menu's Upgrade entry (free) and the demo countdown band (demo); empty on |
| @@ -194,6 +199,9 @@ func main() { |
| case "links": |
| runLinksCmd(os.Args[2:]) |
| return |
| + case "glue": |
| + runGlueCmd(os.Args[2:]) |
| + return |
| case "remove": |
| runRemove(os.Args[2:]) |
| return |
| @@ -319,6 +327,7 @@ func main() { |
| } |
| s := &Server{db: db, hub: hub, pages: pages, changelog: loadChangelog(), version: frontendVersion(), |
| appStoreURL: strings.TrimSpace(os.Getenv("LCHAT_APP_STORE_URL")), homeOrigin: homeOrigin, |
| + glueURL: strings.TrimRight(strings.TrimSpace(os.Getenv("LCHAT_GLUE_URL")), "/"), |
| entitlement: strings.TrimSpace(os.Getenv("LCHAT_ENTITLEMENT")), |
| maxUpload: envBytes("LCHAT_MAX_UPLOAD_BYTES", defaultMaxUploadBytes), |
| blobQuota: envBytes("LCHAT_BLOB_QUOTA_BYTES", defaultBlobQuotaBytes), |
| @@ -747,6 +756,13 @@ func (s *Server) servePage(name string) http.HandlerFunc { |
| // image/frame sources are limited to what the app actually loads (preview |
| // images over https, YouTube's no-cookie embed); everything else is 'self'. |
| nonce := cspNonce() |
| + // The glue (docs/glue.md) is one more named connect-src host, present |
| + // only when this instance opts in — img-src's blanket https: already |
| + // admits its proxied card images. |
| + glueSrc := "" |
| + if s.glueURL != "" { |
| + glueSrc = " " + s.glueURL |
| + } |
| w.Header().Set("Content-Security-Policy", |
| "default-src 'self'; "+ |
| "script-src 'self' 'nonce-"+nonce+"'; "+ |
| @@ -762,7 +778,7 @@ func (s *Server) servePage(name string) http.HandlerFunc { |
| // of the unfurl trade-off. The home origin admits exactly one call: the |
| // enrolment completion poll (backup.js startEnrolPoll), which sends only |
| // the client-random nonce. Keep connect-src pinned to named hosts. |
| - "connect-src 'self' https://*.giphy.com "+s.homeOrigin+"; "+ |
| + "connect-src 'self' https://*.giphy.com "+s.homeOrigin+glueSrc+"; "+ |
| // 'self' admits exactly one same-origin frame: /app-sandbox, the |
| // opaque-origin cage member apps run in (memberapps.go). |
| "frame-src 'self' https://www.youtube-nocookie.com; "+ |
| @@ -787,6 +803,7 @@ func (s *Server) servePage(name string) http.HandlerFunc { |
| page = strings.ReplaceAll(page, "__APP_STORE_URL__", html.EscapeString(s.appStoreURL)) |
| // The home server the chat page enrols into for passkey backup. |
| page = strings.ReplaceAll(page, "__HOME_ORIGIN__", html.EscapeString(s.homeOrigin)) |
| + page = strings.ReplaceAll(page, "__GLUE_URL__", html.EscapeString(s.glueURL)) |
| // The provisioner-pushed plan flag ("free"/"paid"; empty when self-hosted) |
| // — gates the space menu's Upgrade entry, enforced by nothing here: the |
| // checkout itself lives on the home server. |
diff --git a/subcommands.go b/subcommands.go| index 06d67b6..c4df995 100644 |
| --- a/subcommands.go |
| +++ b/subcommands.go |
| @@ -40,6 +40,7 @@ var subcommands = []subcommand{ |
| {"cloud", "run the public self-serve space provisioner"}, |
| {"tunnel", "run the trial-tunnel gateway that fronts `lchat try`"}, |
| {"links", "run the short-link redirector (`links run`)"}, |
| + {"glue", "run the public-web fetch glue: unfurl + signed image proxy (`glue run`)"}, |
| {"hooks", "run the webhook-ingest sidecar (`hooks run`)"}, |
| {"bot", "run the bot sidecar (`bot run`)"}, |
diff --git a/web/pages/index.html b/web/pages/index.html| index 3343d08..3009e6b 100644 |
| --- a/web/pages/index.html |
| +++ b/web/pages/index.html |
| @@ -31,6 +31,9 @@ |
| // demo countdown band (demo) from it — display only, nothing here enforces |
| // billing or the demo TTL. |
| window.__entitlement = "__ENTITLEMENT__"; |
| + // The public-web fetch glue this instance prefers for unfurls (docs/glue.md; |
| + // server-configured via LCHAT_GLUE_URL, empty = unfurl via this instance). |
| + window.__glueURL = "__GLUE_URL__"; |
| </script> |
| <!-- Pre-paint bounce: a signed-out browser visit belongs on home's sign-in |
| page. This must be client-side — the session lives in localStorage, which |
diff --git a/web/static/shell.js b/web/static/shell.js| index 363039b..c333123 100644 |
| --- a/web/static/shell.js |
| +++ b/web/static/shell.js |
| @@ -4924,6 +4924,18 @@ setChatAppHost({ |
| loadBlob: (threadId, att, onProgress) => loadBlob(att.id, threadId, att.type, onProgress), |
| upload: (threadId, file, onProgress) => chatAppUpload(threadId, file, onProgress), |
| unfurl: async (url) => { |
| + // Prefer the shared glue when this instance points at one (docs/glue.md): |
| + // identity-free — no token rides along — and the card's image comes back |
| + // as the glue's signed proxy URL. Any transport failure falls through to |
| + // the instance's own unfurler, so a glue outage degrades to today's path |
| + // (an ok:false glue answer is a real answer — the page had nothing — not |
| + // a reason to re-ask the instance). |
| + if (window.__glueURL) { |
| + try { |
| + const r = await fetch(`${window.__glueURL}/api/unfurl?url=${encodeURIComponent(url)}`); |
| + if (r.ok) return await r.json(); |
| + } catch {} |
| + } |
| try { |
| return await ( |
| await fetch(`/api/unfurl?token=${encodeURIComponent(session.token)}&url=${encodeURIComponent(url)}`) |
diff --git a/web/status/ssh/2026-07-21-glue-dev-standup.json b/web/status/ssh/2026-07-21-glue-dev-standup.json| new file mode 100644 |
| index 0000000..462796c |
| --- /dev/null |
| +++ b/web/status/ssh/2026-07-21-glue-dev-standup.json |
| @@ -0,0 +1,7 @@ |
| +{ |
| + "date": "2026-07-21", |
| + "operator": "Paul Campbell", |
| + "window": "16:40–17:30 UTC (approx)", |
| + "why": "Stood the new public-web fetch glue service up on the Helsinki dev box (branch build, unit + route) and wired a canary space to it for end-to-end testing.", |
| + "refs": ["https://github.com/elevenmessenger/messenger/pull/1241"] |
| +} |