amadan
server: /img shrinks oversized card images to card width (#1253)
shrinkCardImage on the fetch service's image proxy: JPEG/PNG wider than 1600px re-encode at 1200 (CatmullRom, same format, JPEG q80) — og:images are routinely thousands of px wide for a card that renders at a few hundred, and every viewer pays that transfer. GIFs pass through (a resized GIF freezes), as does anything that fails to decode, claims bomb-scale pixel dims (config-checked before any full decode), or comes out bigger 'shrunk' — every failure path returns the original bytes. New dep: golang.org/x/image (official; CatmullRom scaler). docs/fetch.md's client-flip + resize roadmap entries promoted to shipped sections.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
5 files changed,
+148
−19
docs/fetch.md+25 −9fetch.go+54 −0fetch_test.go+56 −0go.mod+3 −2go.sum+10 −8
diff --git a/docs/fetch.md b/docs/fetch.md| index 2e3a853..ebddbe9 100644 |
| --- a/docs/fetch.md |
| +++ b/docs/fetch.md |
| @@ -70,17 +70,33 @@ deployed anywhere yet; when it ships, the planet-scale guide gains its row. |
| No secrets in env — the salt and signing key mint themselves into the DB on |
| first boot. |
| +## The client flip (shipped, env-gated) |
| + |
| +`LCHAT_FETCH_URL` on an instance routes its clients here: the chat page CSP |
| +grows the origin in `connect-src` + `window.__fetchURL` injects it (web), and |
| +`fetch_url` rides `/api/group`'s member-gated read (native — `FeatureGates`, |
| +latched so a failed read never reroutes). Both platforms prefer the service |
| +with instance-unfurl fallback on transport failure. Self-host default stays |
| +*local fetch* — a self-hosted instance silently shipping its members' URLs to |
| +our shared service would betray the self-host story; pointing at the shared |
| +service (or their own) is opt-in config. |
| + |
| +## Resize (shipped) |
| + |
| +`/img` re-encodes oversized JPEG/PNG card images down to card width |
| +(`shrinkCardImage`: >1600px wide → 1200px, CatmullRom, same format) — an |
| +og:image is routinely thousands of px for a card rendering at a few hundred, |
| +and every viewer pays that transfer. GIFs pass through (resizing freezes the |
| +animation), as does anything that fails to decode, claims bomb-scale pixel |
| +dims, or comes out bigger "shrunk" — polish never breaks a card. Public web |
| +images only; member photos keep their client-side pipeline |
| +(`processImage`), always. |
| + |
| ## Roadmap |
| -- **Phase 2 — the client flip.** Instance serves the fetch URL to clients |
| - (config), CSP grows `connect-src`/`img-src` for it, `host.unfurl` prefers |
| - the fetch service with instance-unfurl fallback, and hashcash lands with it. |
| - Self-host default stays *local fetch* — a self-hosted instance silently |
| - shipping its members' URLs to our shared service would betray the self-host story; |
| - pointing at the shared service (or their own) is opt-in config. |
| -- **Resize.** Public card images only (og:image can be multi-MB); a flag on |
| - the `/img` path, never a new trust decision. Member photos keep their |
| - client-side pipeline. |
| +- **Hashcash** before the shared service is load-bearing/default-on for the |
| + hosted fleet (the provisioner's machinery; anonymous callers pay CPU, not |
| + identity). |
| ## Giphy (`GET /api/gif`) |
diff --git a/fetch.go b/fetch.go| index f4bb433..5f5a839 100644 |
| --- a/fetch.go |
| +++ b/fetch.go |
| @@ -21,6 +21,7 @@ |
| package main |
| import ( |
| + "bytes" |
| "crypto/hmac" |
| "crypto/rand" |
| "crypto/sha256" |
| @@ -30,6 +31,9 @@ import ( |
| "encoding/json" |
| "flag" |
| "fmt" |
| + "image" |
| + "image/jpeg" |
| + "image/png" |
| "io" |
| "log" |
| "net" |
| @@ -40,12 +44,18 @@ import ( |
| "strings" |
| "sync" |
| "time" |
| + |
| + xdraw "golang.org/x/image/draw" |
| ) |
| const ( |
| fetchImgMaxBytes = 8 * 1024 * 1024 // largest card image we'll proxy |
| fetchImgTimeout = 15 * time.Second |
| fetchImgCache = "public, max-age=86400" |
| + // Cards render at most a few hundred CSS px wide; og:images routinely |
| + // arrive at several thousand. Anything wider than Over re-encodes at To. |
| + fetchImgResizeOver = 1600 |
| + fetchImgResizeTo = 1200 |
| ) |
| // fetchImgClient fetches card images. Same SSRF posture as ogClient — every |
| @@ -371,12 +381,56 @@ func (g *fetchServer) serveImage(w http.ResponseWriter, r *http.Request) { |
| httpError(w, http.StatusBadGateway, "image too large") |
| return |
| } |
| + body = shrinkCardImage(ct, body) |
| w.Header().Set("Content-Type", resp.Header.Get("Content-Type")) |
| w.Header().Set("Cache-Control", fetchImgCache) |
| w.Header().Set("X-Content-Type-Options", "nosniff") |
| w.Write(body) |
| } |
| +// shrinkCardImage re-encodes an oversized JPEG/PNG down to card width — an |
| +// og:image is often thousands of px wide for a card that renders at a few |
| +// hundred, and every viewer pays that transfer. Only those two formats: a |
| +// resized GIF would freeze its animation, and nothing else here has a stdlib |
| +// encoder. Every failure path returns the original bytes — polish must never |
| +// break a card — and a "shrink" that came out bigger keeps the original too. |
| +func shrinkCardImage(ct string, body []byte) []byte { |
| + if !strings.HasPrefix(ct, "image/jpeg") && !strings.HasPrefix(ct, "image/png") { |
| + return body |
| + } |
| + cfg, kind, err := image.DecodeConfig(bytes.NewReader(body)) |
| + if err != nil || cfg.Width <= fetchImgResizeOver { |
| + return body |
| + } |
| + // Decompression bomb: a tiny file can claim enormous pixel dims. Don't |
| + // decode those ourselves — pass the bytes through and let the browser's |
| + // own limits deal with it. |
| + if cfg.Height <= 0 || cfg.Width*cfg.Height > 40<<20 { |
| + return body |
| + } |
| + src, _, err := image.Decode(bytes.NewReader(body)) |
| + if err != nil { |
| + return body |
| + } |
| + h := cfg.Height * fetchImgResizeTo / cfg.Width |
| + if h < 1 { |
| + h = 1 |
| + } |
| + dst := image.NewRGBA(image.Rect(0, 0, fetchImgResizeTo, h)) |
| + xdraw.CatmullRom.Scale(dst, dst.Bounds(), src, src.Bounds(), xdraw.Over, nil) |
| + var out bytes.Buffer |
| + switch kind { |
| + case "png": |
| + err = png.Encode(&out, dst) |
| + default: |
| + err = jpeg.Encode(&out, dst, &jpeg.Options{Quality: 80}) |
| + } |
| + if err != nil || out.Len() >= len(body) { |
| + return body |
| + } |
| + return out.Bytes() |
| +} |
| + |
| // giphyAPIBase is a var so tests can stand a fake upstream in. |
| var giphyAPIBase = "https://api.giphy.com/v1/gifs/" |
diff --git a/fetch_test.go b/fetch_test.go| index 38a91b6..2b111f3 100644 |
| --- a/fetch_test.go |
| +++ b/fetch_test.go |
| @@ -1,7 +1,10 @@ |
| package main |
| import ( |
| + "bytes" |
| "encoding/json" |
| + "image" |
| + "image/png" |
| "io" |
| "net/http" |
| "net/http/httptest" |
| @@ -313,3 +316,56 @@ func TestFetchSecretsPersist(t *testing.T) { |
| t.Fatalf("weak secrets: key %d bytes, salt %q", len(key), salt) |
| } |
| } |
| + |
| +// Oversized JPEG/PNG card images shrink to card width on the way through the |
| +// proxy; GIFs (animation) and already-small images pass through untouched. |
| +func TestFetchImageProxyShrinks(t *testing.T) { |
| + ts, g := newTestFetch(t) |
| + |
| + allowPrivatePreviewHosts = true |
| + t.Cleanup(func() { allowPrivatePreviewHosts = false }) |
| + |
| + // A 2400×1200 PNG (well over fetchImgResizeOver) and a small one. |
| + big := image.NewRGBA(image.Rect(0, 0, 2400, 1200)) |
| + var bigBuf bytes.Buffer |
| + png.Encode(&bigBuf, big) |
| + var smallBuf bytes.Buffer |
| + png.Encode(&smallBuf, image.NewRGBA(image.Rect(0, 0, 320, 200))) |
| + |
| + site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + w.Header().Set("Content-Type", "image/png") |
| + if r.URL.Path == "/big.png" { |
| + w.Write(bigBuf.Bytes()) |
| + } else { |
| + w.Write(smallBuf.Bytes()) |
| + } |
| + })) |
| + t.Cleanup(site.Close) |
| + |
| + get := func(path string) []byte { |
| + t.Helper() |
| + req, _ := http.NewRequest("GET", ts.URL+"/api/unfurl", nil) |
| + r, err := http.Get(g.proxyImageURL(&http.Request{Host: strings.TrimPrefix(ts.URL, "http://"), URL: req.URL}, site.URL+path)) |
| + if err != nil { |
| + t.Fatalf("proxy %s: %v", path, err) |
| + } |
| + defer r.Body.Close() |
| + b, _ := io.ReadAll(r.Body) |
| + if r.StatusCode != http.StatusOK { |
| + t.Fatalf("proxy %s: status %d", path, r.StatusCode) |
| + } |
| + return b |
| + } |
| + |
| + shrunk := get("/big.png") |
| + cfg, _, err := image.DecodeConfig(bytes.NewReader(shrunk)) |
| + if err != nil || cfg.Width != fetchImgResizeTo || cfg.Height != 600 { |
| + t.Fatalf("big.png came back %dx%d err=%v (want %dx600)", cfg.Width, cfg.Height, err, fetchImgResizeTo) |
| + } |
| + if len(shrunk) >= bigBuf.Len() { |
| + t.Fatalf("shrink didn't shrink: %d >= %d", len(shrunk), bigBuf.Len()) |
| + } |
| + if got := get("/small.png"); string(got) != smallBuf.String() { |
| + t.Fatal("small image should pass through byte-identical") |
| + } |
| +} |
diff --git a/go.mod b/go.mod| index d5ab4dd..0aa274f 100644 |
| --- a/go.mod |
| +++ b/go.mod |
| @@ -17,6 +17,7 @@ require ( |
| github.com/go-webauthn/webauthn v0.11.2 |
| github.com/sqids/sqids-go v0.4.1 |
| golang.org/x/crypto v0.52.0 |
| + golang.org/x/image v0.44.0 |
| golang.org/x/term v0.45.0 |
| modernc.org/sqlite v1.44.3 |
| ) |
| @@ -69,9 +70,9 @@ require ( |
| github.com/x448/float16 v0.8.4 // indirect |
| golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect |
| golang.org/x/net v0.55.0 // indirect |
| - golang.org/x/sync v0.20.0 // indirect |
| + golang.org/x/sync v0.22.0 // indirect |
| golang.org/x/sys v0.47.0 // indirect |
| - golang.org/x/text v0.37.0 // indirect |
| + golang.org/x/text v0.40.0 // indirect |
| google.golang.org/protobuf v1.36.10 // indirect |
| modernc.org/libc v1.67.6 // indirect |
| modernc.org/mathutil v1.7.1 // indirect |
diff --git a/go.sum b/go.sum| index afff804..00b7fed 100644 |
| --- a/go.sum |
| +++ b/go.sum |
| @@ -207,13 +207,15 @@ golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988= |
| golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc= |
| golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2wtOnGAHZWCHUM4KGzY= |
| golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546/go.mod h1:j/pmGrbnkbPtQfxEe5D0VQhZC6qKbfKifgD0oM7sR70= |
| +golang.org/x/image v0.44.0 h1:+tDekMZED9+LrtB3G5xzRggpVh9CARjZqROla3R3R+I= |
| +golang.org/x/image v0.44.0/go.mod h1:V8K3KE9KKKE+pLpQDOeN18w9oacNSvy1tDOirTu4xtY= |
| golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= |
| golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= |
| golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= |
| golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= |
| golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= |
| -golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM= |
| -golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU= |
| +golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= |
| +golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= |
| golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= |
| golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= |
| golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= |
| @@ -233,8 +235,8 @@ golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= |
| golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= |
| golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= |
| golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= |
| -golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= |
| -golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= |
| +golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= |
| +golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= |
| golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= |
| golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= |
| golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= |
| @@ -269,8 +271,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE= |
| golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= |
| golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= |
| golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ= |
| -golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc= |
| -golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38= |
| +golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs= |
| +golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY= |
| golang.org/x/time v0.7.0 h1:ntUhktv3OPE6TgYxXWv9vKvUSJyIFJlyohwbkEwPrKQ= |
| golang.org/x/time v0.7.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= |
| golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= |
| @@ -279,8 +281,8 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc |
| golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= |
| golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58= |
| golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= |
| -golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c= |
| -golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI= |
| +golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q= |
| +golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA= |
| golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= |
| golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= |
| google.golang.org/api v0.155.0 h1:vBmGhCYs0djJttDNynWo44zosHlPvHmA0XiN2zP2DtA= |