eleven / messenger Public

server+web: /gif rides the fetch service — platform Giphy key, media via the signed proxy (#1249)

The fetch service gains GET /api/gif: Giphy search/trending fronted with the
PLATFORM's key (LCHAT_GIPHY_KEY, the unit's one env-file secret), response
passed through in Giphy's own shape with every giphy-hosted media URL
rewritten to the signed /img proxy. hostBridge.gif prefers it when the
instance names a fetch service — no space key needed, so the /gif
get-an-API-key wall disappears — and falls back to the space's
bring-your-own-key direct path on 404 (no upstream key) or any transport
failure. bytes() accepts the fetch service's origin alongside giphy.com.

Trust posture stated in docs/fetch.md § Giphy: the service sees search terms
identity-free and unlogged; Giphy sees only the service's IP; the browser
never talks to Giphy at all on this path. The chosen GIF still encrypts
client-side and posts as an ordinary opaque attachment.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Paul Campbell 3 months ago e585830d61cbc2565f25a070315740244658e71b parent 6595a2a

6 files changed, +237 −9

CLAUDE.md +5 −0
deploy/systemd/lchat-fetch.service +4 −2
docs/fetch.md +18 −5
fetch.go +107 −0
fetch_test.go +80 −0
web/static/shell.js +23 −2

diff --git a/CLAUDE.md b/CLAUDE.md

index 6ecf410..61ddd6a 100644

--- a/CLAUDE.md

+++ b/CLAUDE.md

@@ -45,6 +45,11 @@ Two deliberate exceptions, both easy to gate behind a setting later:

only ever holds the ciphertext. This is the reference for the

**implicitly space-placed** shape (`places: ["space"]` + `listed: false`) and

for space-shared app config generally: `docs/app-credentials.md`.

+ 🤖 When the instance points at a **fetch service** (`LCHAT_FETCH_URL`),

+ search prefers its `/api/gif` instead — platform key, no space key needed,

+ media riding the signed `/img` proxy so the browser never talks to Giphy;

+ the space-key direct path stays as the fallback/self-host story

+ (`docs/fetch.md § Giphy`).

Instance **branding** is plaintext by design and doesn't count against the rule —

it's admin-set, never user content. Three things: the **group name** (`config`

diff --git a/deploy/systemd/lchat-fetch.service b/deploy/systemd/lchat-fetch.service

index c2496d9..656b68c 100644

--- a/deploy/systemd/lchat-fetch.service

+++ b/deploy/systemd/lchat-fetch.service

@@ -4,8 +4,10 @@ After=network-online.target

Wants=network-online.target

[Service]

-# No env file: the service's secrets (preview salt + image-signing key) mint

-# themselves into its DB on first boot (fetch.go); nothing else to configure.

+# The service's own secrets (preview salt + image-signing key) mint themselves

+# into its DB on first boot (fetch.go). The optional env file carries exactly

+# one thing: LCHAT_GIPHY_KEY, the platform Giphy key that turns /api/gif on.

+EnvironmentFile=-/data/fetch.env

ExecStart=/opt/lchat/lchat fetch run --socket /run/lchat/fetch.sock --db /data/fetch.db

Restart=always

RestartSec=1

diff --git a/docs/fetch.md b/docs/fetch.md

index d280265..2e3a853 100644

--- a/docs/fetch.md

+++ b/docs/fetch.md

@@ -81,8 +81,21 @@ first boot.

- **Resize.** Public card images only (og:image can be multi-MB); a flag on

the `/img` path, never a new trust decision. Member photos keep their

client-side pipeline.

-- **Giphy.** Same service family, authenticated upstream: search rides the

- the fetch service with *our* API key (kills the get-an-API-key wall for spaces), the

- space-supplied key remains as override (docs/app-credentials.md). To be

- honest about the trade: today Giphy search is client→Giphy (we see

- nothing); via the service we see search terms, identity-free.

+## Giphy (`GET /api/gif`)

+The authenticated-upstream sibling: search/trending fronted with the

+**platform's** Giphy key (`LCHAT_GIPHY_KEY` in the unit's env file — the one

+env secret this service has), so a space needs no key of its own and the

+get-an-API-key wall disappears. The response is Giphy's own page shape with

+every giphy-hosted media URL rewritten to the signed `/img` proxy, so the

+browser never talks to Giphy: previews and the posted GIF's bytes all ride

+this service's origin, then encrypt client-side and post as ordinary

+attachments exactly as before.

+Honest trade vs the old path: direct client→Giphy meant *we* saw nothing and

+Giphy saw the member's IP + query; via the service, the service sees search

+terms (identity-free, unlogged) and Giphy sees only the service. No key

+configured → `/api/gif` 404s and the client falls back to the space's

+bring-your-own-key path (docs/app-credentials.md), which remains the

+self-host/override story.

diff --git a/fetch.go b/fetch.go

index a91291a..ad67897 100644

--- a/fetch.go

+++ b/fetch.go

@@ -27,6 +27,7 @@ import (

"database/sql"

"encoding/base64"

"encoding/hex"

+ "encoding/json"

"flag"

"fmt"

"io"

@@ -35,6 +36,7 @@ import (

"net/http"

"net/url"

"os"

+ "strconv"

"strings"

"sync"

"time"

@@ -81,6 +83,10 @@ type fetchServer struct {

// bucket (no client IPs are forwarded, on purpose). See the package doc.

unfurlLimiter *modLimiter

imgLimiter *modLimiter

+ gifLimiter *modLimiter

+ // The platform's Giphy API key (LCHAT_GIPHY_KEY). Empty disables /api/gif

+ // (404), and clients fall back to the space's bring-your-own-key path.

+ giphyKey string

}

func newFetchServer(dbPath string) (*fetchServer, error) {

@@ -116,6 +122,7 @@ func newFetchServer(dbPath string) (*fetchServer, error) {

g := &fetchServer{

db: d,

unfurlLimiter: newModLimiter(30, 30.0/60),

+ gifLimiter: newModLimiter(60, 60.0/60),

imgLimiter: newModLimiter(240, 120.0/60),

}

if g.salt, err = fetchConfigSecret(d, "preview_salt"); err != nil {

@@ -152,6 +159,7 @@ func (g *fetchServer) handler() http.Handler {

mux := http.NewServeMux()

mux.HandleFunc("GET /api/unfurl", g.serveUnfurl)

mux.HandleFunc("GET /img", g.serveImage)

+ mux.HandleFunc("GET /api/gif", g.serveGifSearch)

mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {

fmt.Fprintln(w, "ok")

})

@@ -361,6 +369,102 @@ func (g *fetchServer) serveImage(w http.ResponseWriter, r *http.Request) {

w.Write(body)

}

+// giphyAPIBase is a var so tests can stand a fake upstream in.

+var giphyAPIBase = "https://api.giphy.com/v1/gifs/"

+// serveGifSearch fronts Giphy search/trending with the PLATFORM's API key

+// (docs/fetch.md § Giphy), so a space needs no key of its own. The response is

+// Giphy's own page shape, passed through untouched except that every giphy-

+// hosted media URL is rewritten to this service's signed /img proxy — the

+// browser then never talks to Giphy at all: search terms reach it from this

+// service's IP (identity-free), and GIF bytes come back over our origin.

+// No key configured → 404, and the client falls back to the space's

+// bring-your-own-key path (hostBridge.gif in shell.js).

+func (g *fetchServer) serveGifSearch(w http.ResponseWriter, r *http.Request) {

+ if g.giphyKey == "" {

+ httpError(w, http.StatusNotFound, "no gif upstream")

+ return

+ }

+ if !g.gifLimiter.allow(fetchCallerKey(r)) {

+ httpError(w, http.StatusTooManyRequests, "slow down")

+ return

+ }

+ q := strings.TrimSpace(r.URL.Query().Get("q"))

+ if runes := []rune(q); len(runes) > 100 {

+ q = string(runes[:100])

+ }

+ limit := 24

+ if n, err := strconv.Atoi(r.URL.Query().Get("limit")); err == nil && n >= 1 && n <= 50 {

+ limit = n

+ }

+ up := giphyAPIBase + "trending?api_key=" + url.QueryEscape(g.giphyKey) +

+ "&limit=" + strconv.Itoa(limit) + "&rating=pg-13"

+ if q != "" {

+ up = giphyAPIBase + "search?api_key=" + url.QueryEscape(g.giphyKey) +

+ "&q=" + url.QueryEscape(q) + "&limit=" + strconv.Itoa(limit) + "&rating=pg-13"

+ }

+ req, err := http.NewRequestWithContext(r.Context(), "GET", up, nil)

+ if err != nil {

+ httpError(w, http.StatusBadGateway, "gif search failed")

+ return

+ }

+ resp, err := fetchImgClient.Do(req)

+ if err != nil {

+ httpError(w, http.StatusBadGateway, "gif search failed")

+ return

+ }

+ defer resp.Body.Close()

+ if resp.StatusCode != http.StatusOK {

+ httpError(w, http.StatusBadGateway, "gif search failed")

+ return

+ }

+ var page any

+ if json.NewDecoder(io.LimitReader(resp.Body, 4*1024*1024)).Decode(&page) != nil {

+ httpError(w, http.StatusBadGateway, "gif search failed")

+ return

+ }

+ // Results are transient search pages — never cache them (the media URLs

+ // inside are signed against this service's key anyway).

+ w.Header().Set("Cache-Control", "no-store")

+ writeJSON(w, http.StatusOK, g.signGiphyMedia(r, page))

+// signGiphyMedia walks a decoded Giphy page and swaps every giphy-hosted URL

+// string for this service's signed /img URL, wherever it sits — Giphy nests

+// renditions several levels deep and adds new ones over time, so a generic

+// walk beats a field list. Only giphy hosts are signed: other strings (ids,

+// titles, third-party "source" pages we'd never want proxyable) pass through.

+func (g *fetchServer) signGiphyMedia(r *http.Request, v any) any {

+ switch t := v.(type) {

+ case map[string]any:

+ for k, vv := range t {

+ t[k] = g.signGiphyMedia(r, vv)

+ }

+ return t

+ case []any:

+ for i, vv := range t {

+ t[i] = g.signGiphyMedia(r, vv)

+ }

+ return t

+ case string:

+ ok := func(s string) bool { return s == "https" || (s == "http" && allowPrivatePreviewHosts) }

+ if u, err := url.Parse(t); err == nil && ok(u.Scheme) && giphyMediaHost(u.Hostname()) {

+ return g.proxyImageURL(r, t)

+ }

+ return t

+ default:

+ return t

+ }

+func giphyMediaHost(h string) bool {

+ if allowPrivatePreviewHosts {

+ return true // tests: the fake upstream's media lives on loopback

+ }

+ h = strings.ToLower(h)

+ return h == "giphy.com" || strings.HasSuffix(h, ".giphy.com")

// runFetchCmd dispatches `lchat fetch <command>`, mirroring runLinksCmd.

func runFetchCmd(args []string) {

if len(args) < 1 {

@@ -390,6 +494,9 @@ func runFetchRun(args []string) {

if err != nil {

die("fetch: %v", err)

}

+ // The one env secret: the platform Giphy key (unit env file /data/fetch.env).

+ // Absent, /api/gif 404s and clients use the space's own key instead.

+ g.giphyKey = strings.TrimSpace(os.Getenv("LCHAT_GIPHY_KEY"))

ln, addrDesc, err := listen(*addr, *socket)

if err != nil {

die("fetch: %v", err)

diff --git a/fetch_test.go b/fetch_test.go

index 5c92f75..ce2c047 100644

--- a/fetch_test.go

+++ b/fetch_test.go

@@ -207,6 +207,86 @@ func TestFetchClientFlipInjection(t *testing.T) {

// Service secrets survive a restart: the same DB yields the same salt and

// signing key, so cached rows still open and signed URLs stay valid.

+// /api/gif fronts Giphy with the platform key: keyless → 404 (clients fall

+// back to bring-your-own-key), keyed → the upstream page passes through with

+// every media URL rewritten to the signed /img proxy, which then serves the

+// bytes — the browser never needs Giphy.

+func TestFetchGifSearch(t *testing.T) {

+ ts, g := newTestFetch(t)

+ // Keyless: a clean 404 so the client bridge falls through.

+ r, err := http.Get(ts.URL + "/api/gif?q=cats")

+ if err != nil {

+ t.Fatalf("gif keyless: %v", err)

+ }

+ r.Body.Close()

+ if r.StatusCode != http.StatusNotFound {

+ t.Fatalf("gif keyless: want 404, got %d", r.StatusCode)

+ }

+ allowPrivatePreviewHosts = true

+ t.Cleanup(func() { allowPrivatePreviewHosts = false })

+ gifBytes := []byte("GIF89a-not-really")

+ var upstreamQuery url.Values

+ upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

+ if r.URL.Path == "/media/cat.gif" {

+ w.Header().Set("Content-Type", "image/gif")

+ w.Write(gifBytes)

+ return

+ }

+ upstreamQuery = r.URL.Query()

+ w.Header().Set("Content-Type", "application/json")

+ json.NewEncoder(w).Encode(map[string]any{

+ "data": []any{map[string]any{

+ "id": "abc", "title": "Cat",

+ "images": map[string]any{

+ "downsized": map[string]any{

+ "url": "http://" + r.Host + "/media/cat.gif", "size": "1234",

+ },

+ }},

+ })

+ }))

+ t.Cleanup(upstream.Close)

+ oldBase := giphyAPIBase

+ giphyAPIBase = upstream.URL + "/v1/gifs/"

+ t.Cleanup(func() { giphyAPIBase = oldBase })

+ g.giphyKey = "platform-key"

+ r, err = http.Get(ts.URL + "/api/gif?q=cats&limit=5")

+ if err != nil {

+ t.Fatalf("gif search: %v", err)

+ }

+ var page struct {

+ Data []struct {

+ Images map[string]struct{ URL string `json:"url"` } `json:"images"`

+ } `json:"data"`

+ }

+ json.NewDecoder(r.Body).Decode(&page)

+ r.Body.Close()

+ if upstreamQuery.Get("api_key") != "platform-key" || upstreamQuery.Get("q") != "cats" ||

+ upstreamQuery.Get("limit") != "5" {

+ t.Fatalf("upstream saw %v", upstreamQuery)

+ }

+ if len(page.Data) != 1 {

+ t.Fatalf("data = %+v", page.Data)

+ }

+ signed := page.Data[0].Images["downsized"].URL

+ if !strings.HasPrefix(signed, ts.URL+"/img?") {

+ t.Fatalf("media URL not signed onto the proxy: %q", signed)

+ }

+ ir, err := http.Get(signed)

+ if err != nil {

+ t.Fatalf("proxied gif: %v", err)

+ }

+ defer ir.Body.Close()

+ got, _ := io.ReadAll(ir.Body)

+ if ir.StatusCode != http.StatusOK || string(got) != string(gifBytes) {

+ t.Fatalf("proxied gif: status %d, bytes %q", ir.StatusCode, got)

+ }

func TestFetchSecretsPersist(t *testing.T) {

dir := t.TempDir()

g1, err := newFetchServer(dir + "/fetch.db")

diff --git a/web/static/shell.js b/web/static/shell.js

index a969cee..6cd0631 100644

--- a/web/static/shell.js

+++ b/web/static/shell.js

@@ -4725,6 +4725,23 @@ setHostBridge({

// has no key yet), "auth" (Giphy rejected it), null (network/other), so the

// panel can ask for a key exactly when one is the problem.

async search(q, key, limit = 24) {

+ // The shared fetch service searches with the platform's key (docs/fetch.md

+ // § Giphy) — no space key needed, and its results come back with every

+ // media URL already signed onto the service's own /img proxy, so the

+ // browser never talks to Giphy at all. 404 = the service has no upstream

+ // key; that and any transport failure fall through to the space's

+ // bring-your-own-key path below.

+ if (window.__fetchURL) {

+ try {

+ const r = await fetch(

+ `${window.__fetchURL}/api/gif?q=${encodeURIComponent(q || "")}&limit=${limit}`,

+ { signal: AbortSignal.timeout(15000) },

+ );

+ if (r.ok) return (await r.json()).data || [];

+ } catch (e) {

+ console.error("gif search via fetch service failed:", e);

+ }

if (!key) return "key";

const base = "https://api.giphy.com/v1/gifs/";

const url = q

@@ -4746,8 +4763,12 @@ setHostBridge({

async bytes(url, maxBytes) {

try {

const u = new URL(url);

- if (u.protocol !== "https:" || !(u.hostname === "giphy.com" || u.hostname.endsWith(".giphy.com"))) {

- console.error("gif bytes refused non-giphy url");

+ // Two allowed sources, matching the two search paths: the fetch

+ // service's signed /img proxy, or Giphy directly (space-key path).

+ const viaFetch = window.__fetchURL && u.origin === new URL(window.__fetchURL).origin;

+ const giphy = u.protocol === "https:" && (u.hostname === "giphy.com" || u.hostname.endsWith(".giphy.com"));

+ if (!viaFetch && !giphy) {

+ console.error("gif bytes refused url outside giphy/fetch-service");

return null;

}

const r = await fetch(u, { signal: AbortSignal.timeout(30000) });