amadan
server+web+native: hashcash admission for the fetch service (off by default) (#1254)
fetchpow.go, home/pow.go's scheme kept in step: LCHAT_FETCH_POW_BITS arms /api/unfurl + /api/gif behind proof of work — a 401 hands out a sealed single-use challenge, one SHA-256 grind buys a sealed ~1h ticket, and every request after rides (and is rate-bucketed by) that ticket. One solve per client per hour, never per unfurl. Query-param transport so cross-origin callers pay no CORS preflight (the fresh ticket returns in X-Fetch-Ticket, exposed via Access-Control-Expose-Headers); /img stays ungated — its HMAC signature is already the admission. Solvers on both platforms: shell.js fetchServiceGET (async digests, UI keeps breathing) and LChatAPI.fetchServiceGET (grind runs detached). Native also gains the FetchService registry — LChatAPI is a value type built in several places (ThreadStore's computed property, ChatFeedModel's own copy), so the per-instance service URL and earned tickets live process-wide. Also fixes the #1252 native build break this uncovered: ThreadStore.api is a COMPUTED property, so the gates assignment onto it never compiled — the latched URL now lives on the store and threads through api's construction.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
7 files changed,
+507
−37
docs/fetch.md+16 −15fetch.go+88 −4fetch_test.go+68 −0fetchpow.go+173 −0ios/LChatCore/Sources/LChatCore/LChatAPI.swift+99 −9ios/LChatCore/Sources/LChatCore/ThreadStore.swift+16 −2web/static/shell.js+47 −7
diff --git a/docs/fetch.md b/docs/fetch.md| index ebddbe9..a3f7e44 100644 |
| --- a/docs/fetch.md |
| +++ b/docs/fetch.md |
| @@ -41,7 +41,7 @@ instance. Moving the fetch to the shared service buys, in rough order of value: |
| dictionaries. The cache at rest reveals nothing a reader doesn't already |
| know the URL for. No request logs; the only log line is startup. |
| - **Never:** member content. Member photos/files are encrypted client-side |
| - (`processImage` + `encryptBytes`) and are categorically out of scope — the |
| + (`processImage` + `encryptBytes`) and are categorically out of scope — |
| its remit is content already on the public web. Any future endpoint that |
| would touch member bytes is a design error; say no. |
| - **Not an open proxy:** `/img` serves only URLs the service itself HMAC-signed |
| @@ -52,14 +52,21 @@ instance. Moving the fetch to the shared service buys, in rough order of value: |
| re-validates the resolved IP via `guardDialControl` — same guard as the |
| instance, links, and push. |
| -## Abuse posture (v1: honest, modest) |
| - |
| -`/api/unfurl` and `/img` carry per-IP token buckets, which behind the router |
| -collapse into global buckets (no forwarded IPs — a privacy feature with a |
| -rate-limiting cost). That is acceptable while nothing points at it by |
| -default. Before the client flip makes it load-bearing, add **hashcash** (the |
| -cloud provisioner already ships the machinery) so anonymous callers pay in |
| -CPU instead of identity. |
| +## Abuse posture — hashcash admission (shipped, off by default) |
| + |
| +Set `LCHAT_FETCH_POW_BITS` (0/unset = off) and `/api/unfurl` + `/api/gif` |
| +demand proof of work — home's space-creation scheme (`fetchpow.go`, kept in |
| +step with `home/pow.go`): a 401 hands out a sealed single-use challenge, |
| +the client grinds SHA-256 to that many leading zero bits ONCE, the solved |
| +request is served and answered with a sealed ~1h **ticket**, and every call |
| +after rides (and is rate-bucketed by) the ticket — one solve per client per |
| +hour, never per unfurl. Admission travels in query params so cross-origin |
| +callers pay no CORS preflight; `/img` needs none (its HMAC signature is the |
| +admission). Both web (`fetchServiceGET`, shell.js) and native |
| +(`LChatAPI.fetchServiceGET` + the `FetchService` registry) solve and hold |
| +tickets. Ungated, the per-IP buckets remain — which behind the router |
| +collapse into global buckets (no forwarded IPs — a privacy feature): arm |
| +the gate before the shared service is load-bearing for the hosted fleet. |
| ## Deployment |
| @@ -92,12 +99,6 @@ dims, or comes out bigger "shrunk" — polish never breaks a card. Public web |
| images only; member photos keep their client-side pipeline |
| (`processImage`), always. |
| -## Roadmap |
| - |
| -- **Hashcash** before the shared service is load-bearing/default-on for the |
| - hosted fleet (the provisioner's machinery; anonymous callers pay CPU, not |
| - identity). |
| - |
| ## Giphy (`GET /api/gif`) |
| The authenticated-upstream sibling: search/trending fronted with the |
diff --git a/fetch.go b/fetch.go| index 5f5a839..36b98b2 100644 |
| --- a/fetch.go |
| +++ b/fetch.go |
| @@ -97,6 +97,10 @@ type fetchServer struct { |
| // The platform's Giphy API key (LCHAT_GIPHY_KEY). Empty disables /api/gif |
| // (404), and clients fall back to the space's bring-your-own-key path. |
| giphyKey string |
| + // Hashcash admission (fetchpow.go); nil = off (the default). Its seal key |
| + // is minted into the DB alongside the salt so restarts keep tickets valid. |
| + pow *fetchPowGate |
| + powSealKey []byte |
| } |
| func newFetchServer(dbPath string) (*fetchServer, error) { |
| @@ -145,9 +149,26 @@ func newFetchServer(dbPath string) (*fetchServer, error) { |
| if g.signKey, err = hex.DecodeString(keyHex); err != nil { |
| return nil, err |
| } |
| + powHex, err := fetchConfigSecret(d, "pow_seal_key") |
| + if err != nil { |
| + return nil, err |
| + } |
| + if g.powSealKey, err = hex.DecodeString(powHex); err != nil { |
| + return nil, err |
| + } |
| return g, nil |
| } |
| +// armPow turns the hashcash gate on at the given difficulty (0 = off). |
| +func (g *fetchServer) armPow(bits int) error { |
| + gate, err := newFetchPowGate(bits, g.powSealKey) |
| + if err != nil { |
| + return err |
| + } |
| + g.pow = gate |
| + return nil |
| +} |
| + |
| // fetchConfigSecret returns the named config value, minting a random one on |
| // first boot — the same insert-if-absent pattern as the instance's |
| // previewSalt, so restarts and concurrent starts converge on one value. |
| @@ -167,9 +188,11 @@ func fetchConfigSecret(d *sql.DB, key string) (string, error) { |
| func (g *fetchServer) handler() http.Handler { |
| mux := http.NewServeMux() |
| - mux.HandleFunc("GET /api/unfurl", g.serveUnfurl) |
| + // The two open endpoints sit behind the (optional) hashcash gate; /img |
| + // needs none — its HMAC signature is already the admission. |
| + mux.HandleFunc("GET /api/unfurl", g.powGated(g.serveUnfurl)) |
| mux.HandleFunc("GET /img", g.serveImage) |
| - mux.HandleFunc("GET /api/gif", g.serveGifSearch) |
| + mux.HandleFunc("GET /api/gif", g.powGated(g.serveGifSearch)) |
| mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) { |
| fmt.Fprintln(w, "ok") |
| }) |
| @@ -187,10 +210,63 @@ func (g *fetchServer) handler() http.Handler { |
| // design — no cookies, no credentials, nothing origin-scoped to protect. |
| return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| w.Header().Set("Access-Control-Allow-Origin", "*") |
| + // The pow gate hands fresh tickets back on a response header; without |
| + // this, cross-origin JS can see the response but not that header. |
| + w.Header().Set("Access-Control-Expose-Headers", "X-Fetch-Ticket") |
| mux.ServeHTTP(w, r) |
| }) |
| } |
| +// powGated wraps an open endpoint in the hashcash gate (nil gate = pass). |
| +// Admission rides query params, not headers — a custom request header would |
| +// cost every cross-origin caller a CORS preflight round trip. |
| +// |
| +// - ?ticket= a sealed, unexpired ticket → in, rate-bucketed by the ticket |
| +// - ?pow=&nonce= a solved challenge → consumed, request served, and a |
| +// fresh ticket returned in X-Fetch-Ticket for the requests that follow |
| +// - neither → 401 {"pow":{"challenge","bits"}} — the mint rides the |
| +// rejection, so admission costs one round trip, not two |
| +func (g *fetchServer) powGated(next http.HandlerFunc) http.HandlerFunc { |
| + return func(w http.ResponseWriter, r *http.Request) { |
| + if g.pow == nil { |
| + next(w, r) |
| + return |
| + } |
| + q := r.URL.Query() |
| + if t := q.Get("ticket"); t != "" && g.pow.ticketOK(t) { |
| + next(w, r) |
| + return |
| + } |
| + if c := q.Get("pow"); c != "" && g.pow.verifyAndConsume(c, q.Get("nonce")) == nil { |
| + if t, err := g.pow.mintTicket(); err == nil { |
| + w.Header().Set("X-Fetch-Ticket", t) |
| + } |
| + next(w, r) |
| + return |
| + } |
| + ch, err := g.pow.mintChallenge() |
| + if err != nil { |
| + httpError(w, http.StatusInternalServerError, "pow mint failed") |
| + return |
| + } |
| + writeJSON(w, http.StatusUnauthorized, map[string]any{ |
| + "pow": map[string]any{"challenge": ch, "bits": g.pow.bits}, |
| + }) |
| + } |
| +} |
| + |
| +// rateKey buckets a gated request by its ticket (each solved client gets its |
| +// own bucket); ungated requests keep the caller-IP key, with its documented |
| +// behind-the-router collapse. |
| +func (g *fetchServer) rateKey(r *http.Request) string { |
| + if g.pow != nil { |
| + if t := r.URL.Query().Get("ticket"); t != "" { |
| + return "t:" + t |
| + } |
| + } |
| + return fetchCallerKey(r) |
| +} |
| + |
| // fetchCallerKey is the rate-limit bucket key. Mirrors linkscmd.go — behind |
| // the router (unix socket, no forwarded IPs) this degrades to one shared |
| // bucket, which is the accepted posture for now (package doc). |
| @@ -206,7 +282,7 @@ func fetchCallerKey(r *http.Request) string { |
| // validation, same language handling, same stale-serve semantics — but the |
| // card's image URL is rewritten to the signed /img proxy before it leaves. |
| func (g *fetchServer) serveUnfurl(w http.ResponseWriter, r *http.Request) { |
| - if !g.unfurlLimiter.allow(fetchCallerKey(r)) { |
| + if !g.unfurlLimiter.allow(g.rateKey(r)) { |
| httpError(w, http.StatusTooManyRequests, "slow down") |
| return |
| } |
| @@ -447,7 +523,7 @@ func (g *fetchServer) serveGifSearch(w http.ResponseWriter, r *http.Request) { |
| httpError(w, http.StatusNotFound, "no gif upstream") |
| return |
| } |
| - if !g.gifLimiter.allow(fetchCallerKey(r)) { |
| + if !g.gifLimiter.allow(g.rateKey(r)) { |
| httpError(w, http.StatusTooManyRequests, "slow down") |
| return |
| } |
| @@ -559,6 +635,14 @@ func runFetchRun(args []string) { |
| // The one env secret: the platform Giphy key (unit env file /data/fetch.env). |
| // Absent, /api/gif 404s and clients use the space's own key instead. |
| g.giphyKey = strings.TrimSpace(os.Getenv("LCHAT_GIPHY_KEY")) |
| + // Hashcash difficulty (LCHAT_FETCH_POW_BITS, 0/unset = off): arm before |
| + // pointing the hosted fleet here; leave off for dev and self-hosters. |
| + if bits, _ := strconv.Atoi(strings.TrimSpace(os.Getenv("LCHAT_FETCH_POW_BITS"))); bits > 0 { |
| + if err := g.armPow(bits); err != nil { |
| + die("fetch: pow: %v", err) |
| + } |
| + log.Printf("fetch: hashcash admission armed (%d bits)", bits) |
| + } |
| ln, addrDesc, err := listen(*addr, *socket) |
| if err != nil { |
| die("fetch: %v", err) |
diff --git a/fetch_test.go b/fetch_test.go| index 2b111f3..b6dd9d7 100644 |
| --- a/fetch_test.go |
| +++ b/fetch_test.go |
| @@ -369,3 +369,71 @@ func TestFetchImageProxyShrinks(t *testing.T) { |
| t.Fatal("small image should pass through byte-identical") |
| } |
| } |
| + |
| +// The hashcash gate: unadmitted requests get a challenge, a solved challenge |
| +// serves the request and mints a reusable ticket, tickets admit until expiry, |
| +// and a solved challenge can't be replayed for a second ticket. |
| +func TestFetchPowGate(t *testing.T) { |
| + ts, g := newTestFetch(t) |
| + if err := g.armPow(8); err != nil { // 8 bits: a few hundred hashes |
| + t.Fatalf("armPow: %v", err) |
| + } |
| + |
| + allowPrivatePreviewHosts = true |
| + t.Cleanup(func() { allowPrivatePreviewHosts = false }) |
| + site := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + w.Header().Set("Content-Type", "text/html") |
| + w.Write([]byte(`<head><meta property="og:title" content="Gated"></head>`)) |
| + })) |
| + t.Cleanup(site.Close) |
| + target := "/api/unfurl?url=" + url.QueryEscape(site.URL) |
| + |
| + // No ticket, no pow → 401 with a challenge. |
| + r, err := http.Get(ts.URL + target) |
| + if err != nil { |
| + t.Fatalf("gated: %v", err) |
| + } |
| + var gate struct { |
| + Pow struct { |
| + Challenge string `json:"challenge"` |
| + Bits int `json:"bits"` |
| + } `json:"pow"` |
| + } |
| + json.NewDecoder(r.Body).Decode(&gate) |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusUnauthorized || gate.Pow.Challenge == "" || gate.Pow.Bits != 8 { |
| + t.Fatalf("gate = %d %+v", r.StatusCode, gate) |
| + } |
| + |
| + // Solve → the request itself is served AND a ticket comes back. |
| + nonce := fetchSolvePow(gate.Pow.Challenge, gate.Pow.Bits) |
| + r, err = http.Get(ts.URL + target + "&pow=" + url.QueryEscape(gate.Pow.Challenge) + "&nonce=" + nonce) |
| + if err != nil { |
| + t.Fatalf("solved: %v", err) |
| + } |
| + ticket := r.Header.Get("X-Fetch-Ticket") |
| + var pv LinkPreview |
| + json.NewDecoder(r.Body).Decode(&pv) |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusOK || !pv.OK || pv.Title != "Gated" || ticket == "" { |
| + t.Fatalf("solved: status %d pv %+v ticket %q", r.StatusCode, pv, ticket) |
| + } |
| + |
| + // The ticket readmits; the spent challenge does not. |
| + r, _ = http.Get(ts.URL + target + "&ticket=" + url.QueryEscape(ticket)) |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusOK { |
| + t.Fatalf("ticket readmit: %d", r.StatusCode) |
| + } |
| + r, _ = http.Get(ts.URL + target + "&pow=" + url.QueryEscape(gate.Pow.Challenge) + "&nonce=" + nonce) |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusUnauthorized { |
| + t.Fatalf("replayed challenge: want 401, got %d", r.StatusCode) |
| + } |
| + // Garbage tickets bounce back to the gate. |
| + r, _ = http.Get(ts.URL + target + "&ticket=garbage") |
| + r.Body.Close() |
| + if r.StatusCode != http.StatusUnauthorized { |
| + t.Fatalf("garbage ticket: want 401, got %d", r.StatusCode) |
| + } |
| +} |
diff --git a/fetchpow.go b/fetchpow.go| new file mode 100644 |
| index 0000000..bdb301a |
| --- /dev/null |
| +++ b/fetchpow.go |
| @@ -0,0 +1,173 @@ |
| +// Hashcash admission for the fetch service (docs/fetch.md § Abuse posture). |
| +// The service is identity-free by design and, behind the router, can't even |
| +// tell callers apart by IP — so before it becomes load-bearing for the hosted |
| +// fleet, anonymous callers pay in CPU instead of identity: solve one |
| +// challenge, receive a sealed ~1h ticket, and every request rides (and is |
| +// rate-bucketed by) that ticket. One solve per client per hour, not per |
| +// request — an unfurl must stay instant. |
| +// |
| +// The scheme is home/pow.go's (space creation, #561), kept deliberately in |
| +// step: SHA-256(challenge ":" nonce) clearing `bits` leading zero bits, |
| +// sealed expiring single-use challenges, verification costing one hash. Off |
| +// by default (LCHAT_FETCH_POW_BITS unset/0) — self-hosters and dev setups |
| +// never see it. |
| +package main |
| + |
| +import ( |
| + "crypto/aes" |
| + "crypto/cipher" |
| + "crypto/rand" |
| + "crypto/sha256" |
| + "encoding/base64" |
| + "encoding/binary" |
| + "fmt" |
| + "strconv" |
| + "sync" |
| + "time" |
| +) |
| + |
| +const ( |
| + fetchPowChallengeWindow = 5 * time.Minute |
| + fetchPowTicketWindow = time.Hour |
| +) |
| + |
| +// fetchPowGate is the admission gate; a nil *fetchPowGate means off. |
| +type fetchPowGate struct { |
| + bits int |
| + aead cipher.AEAD // sealed challenges + tickets; key minted into the service DB |
| + mu sync.Mutex |
| + used map[string]int64 // consumed challenge -> expiry (replay guard) |
| +} |
| + |
| +func newFetchPowGate(bits int, key []byte) (*fetchPowGate, error) { |
| + if bits <= 0 { |
| + return nil, nil |
| + } |
| + block, err := aes.NewCipher(key) |
| + if err != nil { |
| + return nil, err |
| + } |
| + aead, err := cipher.NewGCM(block) |
| + if err != nil { |
| + return nil, err |
| + } |
| + return &fetchPowGate{bits: bits, aead: aead, used: map[string]int64{}}, nil |
| +} |
| + |
| +// seal/open: base64url(iv||ciphertext), the crypto.js convention like the |
| +// preview cache's. |
| +func (g *fetchPowGate) seal(payload []byte) (string, error) { |
| + iv := make([]byte, g.aead.NonceSize()) |
| + if _, err := rand.Read(iv); err != nil { |
| + return "", err |
| + } |
| + return base64.RawURLEncoding.EncodeToString(g.aead.Seal(iv, iv, payload, nil)), nil |
| +} |
| + |
| +func (g *fetchPowGate) open(blob string) ([]byte, bool) { |
| + raw, err := base64.RawURLEncoding.DecodeString(blob) |
| + if err != nil || len(raw) < g.aead.NonceSize() { |
| + return nil, false |
| + } |
| + payload, err := g.aead.Open(nil, raw[:g.aead.NonceSize()], raw[g.aead.NonceSize():], nil) |
| + if err != nil { |
| + return nil, false |
| + } |
| + return payload, true |
| +} |
| + |
| +// mintChallenge issues one solvable challenge: seal(expiry || random). |
| +// Stateless until solved — only CONSUMED challenges take memory. |
| +func (g *fetchPowGate) mintChallenge() (string, error) { |
| + return g.sealExpiring(time.Now().Add(fetchPowChallengeWindow)) |
| +} |
| + |
| +func (g *fetchPowGate) sealExpiring(expiry time.Time) (string, error) { |
| + payload := make([]byte, 8+16) |
| + binary.BigEndian.PutUint64(payload, uint64(expiry.Unix())) |
| + if _, err := rand.Read(payload[8:]); err != nil { |
| + return "", err |
| + } |
| + return g.seal(payload) |
| +} |
| + |
| +// verifyAndConsume checks one solution: the challenge opens, hasn't expired |
| +// or been used, and the nonce clears the difficulty. A passing challenge is |
| +// consumed — one solve, one ticket. |
| +func (g *fetchPowGate) verifyAndConsume(challenge, nonce string) error { |
| + payload, ok := g.open(challenge) |
| + if !ok || len(payload) != 24 { |
| + return fmt.Errorf("bad challenge") |
| + } |
| + expiry := int64(binary.BigEndian.Uint64(payload)) |
| + if time.Now().Unix() > expiry { |
| + return fmt.Errorf("challenge expired") |
| + } |
| + if !fetchPowSolves(challenge, nonce, g.bits) { |
| + return fmt.Errorf("proof of work does not check out") |
| + } |
| + g.mu.Lock() |
| + defer g.mu.Unlock() |
| + now := time.Now().Unix() |
| + for c, exp := range g.used { // expire the replay guard as we go |
| + if exp < now { |
| + delete(g.used, c) |
| + } |
| + } |
| + if _, replayed := g.used[challenge]; replayed { |
| + return fmt.Errorf("challenge already used") |
| + } |
| + g.used[challenge] = expiry |
| + return nil |
| +} |
| + |
| +// mintTicket issues the post-solve pass: same sealed shape, longer window, |
| +// no replay guard (a ticket is MEANT to be reused; the per-ticket rate |
| +// bucket is what bounds it). |
| +func (g *fetchPowGate) mintTicket() (string, error) { |
| + return g.sealExpiring(time.Now().Add(fetchPowTicketWindow)) |
| +} |
| + |
| +func (g *fetchPowGate) ticketOK(t string) bool { |
| + payload, ok := g.open(t) |
| + if !ok || len(payload) != 24 { |
| + return false |
| + } |
| + return time.Now().Unix() <= int64(binary.BigEndian.Uint64(payload)) |
| +} |
| + |
| +// fetchPowSolves / fetchLeadingZeroBits / fetchSolvePow mirror home/pow.go |
| +// (same package-main symbols exist there, unexported in that sibling binary's |
| +// package — keep the two in step). |
| +func fetchPowSolves(challenge, nonce string, bits int) bool { |
| + h := sha256.Sum256([]byte(challenge + ":" + nonce)) |
| + return fetchLeadingZeroBits(h[:]) >= bits |
| +} |
| + |
| +func fetchLeadingZeroBits(h []byte) int { |
| + bits := 0 |
| + for _, b := range h { |
| + if b == 0 { |
| + bits += 8 |
| + continue |
| + } |
| + for mask := byte(0x80); mask != 0; mask >>= 1 { |
| + if b&mask != 0 { |
| + return bits |
| + } |
| + bits++ |
| + } |
| + } |
| + return bits |
| +} |
| + |
| +// fetchSolvePow grinds nonces until one clears the difficulty — tests (at low |
| +// bits); the web and native clients carry their own solver loops. |
| +func fetchSolvePow(challenge string, bits int) string { |
| + for n := 0; ; n++ { |
| + nonce := strconv.Itoa(n) |
| + if fetchPowSolves(challenge, nonce, bits) { |
| + return nonce |
| + } |
| + } |
| +} |
diff --git a/ios/LChatCore/Sources/LChatCore/LChatAPI.swift b/ios/LChatCore/Sources/LChatCore/LChatAPI.swift| index 6128800..de0f5be 100644 |
| --- a/ios/LChatCore/Sources/LChatCore/LChatAPI.swift |
| +++ b/ios/LChatCore/Sources/LChatCore/LChatAPI.swift |
| @@ -1,3 +1,4 @@ |
| +import CryptoKit |
| import Foundation |
| // Thin REST client for the lchat HTTP API. Every data endpoint is token-gated; the |
| @@ -391,15 +392,17 @@ public struct LChatAPI { |
| let ok: Bool? |
| } |
| var r: Resp? |
| - if !fetchURL.isEmpty, |
| - var c = URLComponents(string: fetchURL + "/api/unfurl") { |
| - c.queryItems = [URLQueryItem(name: "url", value: url.absoluteString)] |
| - if let u = c.url, let (data, resp) = try? await session.data(from: u), |
| - (resp as? HTTPURLResponse)?.statusCode == 200 { |
| - // A decoded answer — even ok:false — is real (the page offered |
| - // nothing); only transport failures fall through to the instance. |
| - r = try? JSONDecoder().decode(Resp.self, from: data) |
| - } |
| + // The explicit field (tests) wins; otherwise the registry entry the |
| + // store latched from FeatureGates — every LChatAPI copy for this |
| + // instance routes alike, however it was constructed. |
| + let svc = !fetchURL.isEmpty ? fetchURL |
| + : FetchService.shared.url(forInstanceHost: baseURL.host ?? "") |
| + if !svc.isEmpty, |
| + let query = url.absoluteString.addingPercentEncoding(withAllowedCharacters: .alphanumerics), |
| + let data = await fetchServiceGET(svc, "/api/unfurl?url=" + query) { |
| + // A decoded answer — even ok:false — is real (the page offered |
| + // nothing); only transport failures fall through to the instance. |
| + r = try? JSONDecoder().decode(Resp.self, from: data) |
| } |
| if r == nil { |
| let items = [ |
| @@ -413,6 +416,64 @@ public struct LChatAPI { |
| description: r.description ?? "", imageURL: URL(string: r.image ?? "")) |
| } |
| + /// One GET against the shared fetch service (docs/fetch.md), handling its |
| + /// optional hashcash admission: a 401 carries {pow:{challenge,bits}} — |
| + /// grind SHA-256(challenge ":" nonce) to `bits` leading zeros off-actor, |
| + /// retry with the solution, and keep the returned ticket (process-wide, |
| + /// FetchTickets) so it's one solve per ~hour, not per request. Query |
| + /// params, matching the web client. nil = transport failure or non-200, |
| + /// which callers treat as "fall back to the instance". |
| + func fetchServiceGET(_ base: String, _ path: String) async -> Data? { |
| + let sep = path.contains("?") ? "&" : "?" |
| + func get(_ extra: String) async -> (Data, HTTPURLResponse)? { |
| + guard let u = URL(string: base + path + extra), |
| + let (data, resp) = try? await session.data(from: u), |
| + let http = resp as? HTTPURLResponse else { return nil } |
| + if let t = http.value(forHTTPHeaderField: "X-Fetch-Ticket"), !t.isEmpty { |
| + FetchService.shared.setTicket(t, for: base) |
| + } |
| + return (data, http) |
| + } |
| + let ticket = FetchService.shared.ticket(for: base) |
| + let extra = ticket.isEmpty ? "" : |
| + sep + "ticket=" + (ticket.addingPercentEncoding(withAllowedCharacters: .alphanumerics) ?? "") |
| + guard var (data, http) = await get(extra) else { return nil } |
| + if http.statusCode == 401 { |
| + struct Gate: Decodable { |
| + struct Pow: Decodable { let challenge: String; let bits: Int } |
| + let pow: Pow? |
| + } |
| + guard let pow = (try? JSONDecoder().decode(Gate.self, from: data))?.pow else { return nil } |
| + let nonce = await Task.detached(priority: .userInitiated) { |
| + var n = 0 |
| + while !LChatAPI.powSolves(pow.challenge, String(n), pow.bits) { n += 1 } |
| + return n |
| + }.value |
| + let solved = sep + "pow=" + |
| + (pow.challenge.addingPercentEncoding(withAllowedCharacters: .alphanumerics) ?? "") + |
| + "&nonce=\(nonce)" |
| + guard let again = await get(solved) else { return nil } |
| + (data, http) = again |
| + } |
| + return http.statusCode == 200 ? data : nil |
| + } |
| + |
| + /// SHA-256(challenge ":" nonce) clears `bits` leading zero bits — the |
| + /// same check as the server's (fetchpow.go) and home's space-creation gate. |
| + static func powSolves(_ challenge: String, _ nonce: String, _ bits: Int) -> Bool { |
| + var count = 0 |
| + for b in SHA256.hash(data: Data((challenge + ":" + nonce).utf8)) { |
| + if b == 0 { count += 8; continue } |
| + var mask: UInt8 = 0x80 |
| + while mask != 0 { |
| + if b & mask != 0 { return count >= bits } |
| + count += 1 |
| + mask >>= 1 |
| + } |
| + } |
| + return count >= bits |
| + } |
| + |
| /// The instance's display name, or "" when it was never named, so callers |
| /// fall back to the host instead of parroting the compiled-in default. |
| /// The server's `name_set` flag is the authority (an admin-chosen name — |
| @@ -781,3 +842,32 @@ public struct LChatAPI { |
| return resp != nil |
| } |
| } |
| + |
| +/// Process-wide fetch-service state: LChatAPI is a value type constructed |
| +/// freely all over (ThreadStore's computed property, ChatFeedModel's own |
| +/// copy, the share extension), so the per-instance service URL learned from |
| +/// FeatureGates — and the hashcash ticket earned against a service — live |
| +/// here, keyed by instance host / service base respectively. |
| +final class FetchService: @unchecked Sendable { |
| + static let shared = FetchService() |
| + private let lock = NSLock() |
| + private var urlByInstanceHost: [String: String] = [:] |
| + private var ticketByBase: [String: String] = [:] |
| + |
| + func url(forInstanceHost host: String) -> String { |
| + lock.lock(); defer { lock.unlock() } |
| + return urlByInstanceHost[host] ?? "" |
| + } |
| + func setURL(_ url: String, forInstanceHost host: String) { |
| + lock.lock(); defer { lock.unlock() } |
| + urlByInstanceHost[host] = url |
| + } |
| + func ticket(for base: String) -> String { |
| + lock.lock(); defer { lock.unlock() } |
| + return ticketByBase[base] ?? "" |
| + } |
| + func setTicket(_ ticket: String, for base: String) { |
| + lock.lock(); defer { lock.unlock() } |
| + ticketByBase[base] = ticket |
| + } |
| +} |
diff --git a/ios/LChatCore/Sources/LChatCore/ThreadStore.swift b/ios/LChatCore/Sources/LChatCore/ThreadStore.swift| index afadeb0..3c157e6 100644 |
| --- a/ios/LChatCore/Sources/LChatCore/ThreadStore.swift |
| +++ b/ios/LChatCore/Sources/LChatCore/ThreadStore.swift |
| @@ -63,9 +63,16 @@ public final class ThreadStore: ObservableObject { |
| // shape, replayed over a fresh server read on every save. |
| private var mutedOps: [String: Bool] = [:] |
| + // The fetch service this instance prefers for unfurls (docs/fetch.md), |
| + // latched from FeatureGates on refresh. Lives here because `api` below is |
| + // computed fresh per access — state set on one copy would evaporate. |
| + private var fetchServiceURL = "" |
| + |
| private var api: LChatAPI? { |
| guard let base = instance.baseURL else { return nil } |
| - return LChatAPI(baseURL: base, token: instance.deviceToken) |
| + var a = LChatAPI(baseURL: base, token: instance.deviceToken) |
| + a.fetchURL = fetchServiceURL |
| + return a |
| } |
| public init(instance: SavedInstance) { |
| @@ -184,7 +191,14 @@ public final class ThreadStore: ObservableObject { |
| // Route unfurls through the instance's preferred fetch service. nil = |
| // the gates read failed — keep the routing we hold (a blip must not |
| // reroute); "" is a real answer (the instance says unfurl locally). |
| - if let fetchURL = gates.fetchURL { api.fetchURL = fetchURL } |
| + // The registry copy is what reaches LChatAPI values built elsewhere |
| + // (ChatFeedModel constructs its own — see FetchService). |
| + if let fetchURL = gates.fetchURL { |
| + fetchServiceURL = fetchURL |
| + if let host = instance.baseURL?.host { |
| + FetchService.shared.setURL(fetchURL, forInstanceHost: host) |
| + } |
| + } |
| // nil = fetch failed — keep the held set rather than "unblocking" |
| // everyone over a network blip. |
| if let blocked = await blocksAsync { blockedPeople = Set(blocked) } |
diff --git a/web/static/shell.js b/web/static/shell.js| index 6cd0631..73b86f2 100644 |
| --- a/web/static/shell.js |
| +++ b/web/static/shell.js |
| @@ -4216,6 +4216,49 @@ async function awaitConnectResult(threadId, key, rid, tries = 24) { |
| let oauthPending = null; // { service, state } |
| const oauthCfgCache = {}; // service -> { client_id, authorize_url, redirect_uri } |
| +// ---- fetch-service doorway (docs/fetch.md) -------------------------------- |
| +// One GET against the shared fetch service, handling its optional hashcash |
| +// admission: a 401 carries {pow:{challenge,bits}} — grind SHA-256(challenge |
| +// ":" nonce) to `bits` leading zeros (async digests yield, so the UI keeps |
| +// breathing), retry with the solution, and keep the returned ticket for every |
| +// call after (one solve per ~hour, not per request). Query params, not |
| +// headers — a custom header would cost a CORS preflight per call. |
| +let fetchServiceTicket = ""; |
| + |
| +function fsLeadingZeroBits(bytes) { |
| + let bits = 0; |
| + for (const b of bytes) { |
| + if (b === 0) { bits += 8; continue; } |
| + for (let mask = 128; mask; mask >>= 1) { |
| + if (b & mask) return bits; |
| + bits++; |
| + } |
| + } |
| + return bits; |
| +} |
| + |
| +async function fetchServiceGET(path) { |
| + const base = window.__fetchURL; |
| + if (!base) return null; |
| + const sep = path.includes("?") ? "&" : "?"; |
| + const get = (extra) => fetch(base + path + extra, { signal: AbortSignal.timeout(15000) }); |
| + let r = await get(fetchServiceTicket ? `${sep}ticket=${encodeURIComponent(fetchServiceTicket)}` : ""); |
| + if (r.status === 401) { |
| + const c = ((await r.json().catch(() => ({}))).pow) || null; |
| + if (!c || !c.challenge) return r; |
| + const enc = new TextEncoder(); |
| + for (let n = 0; ; n++) { |
| + const d = new Uint8Array(await crypto.subtle.digest("SHA-256", enc.encode(`${c.challenge}:${n}`))); |
| + if (fsLeadingZeroBits(d) >= c.bits) { |
| + r = await get(`${sep}pow=${encodeURIComponent(c.challenge)}&nonce=${n}`); |
| + fetchServiceTicket = r.headers.get("X-Fetch-Ticket") || fetchServiceTicket; |
| + break; |
| + } |
| + } |
| + } |
| + return r; |
| +} |
| + |
| let oauthCompleting = false; // true while a returned code is being exchanged (drives the "Connecting…" state) |
| // Fetch + cache the broker's public OAuth config. The connect surface warms this |
| @@ -4733,11 +4776,8 @@ setHostBridge({ |
| // bring-your-own-key path below. |
| if (window.__fetchURL) { |
| try { |
| - const r = await fetch( |
| - `${window.__fetchURL}/api/gif?q=${encodeURIComponent(q || "")}&limit=${limit}`, |
| - { signal: AbortSignal.timeout(15000) }, |
| - ); |
| - if (r.ok) return (await r.json()).data || []; |
| + const r = await fetchServiceGET(`/api/gif?q=${encodeURIComponent(q || "")}&limit=${limit}`); |
| + if (r && r.ok) return (await r.json()).data || []; |
| } catch (e) { |
| console.error("gif search via fetch service failed:", e); |
| } |
| @@ -4953,8 +4993,8 @@ setChatAppHost({ |
| // a reason to re-ask the instance). |
| if (window.__fetchURL) { |
| try { |
| - const r = await fetch(`${window.__fetchURL}/api/unfurl?url=${encodeURIComponent(url)}`); |
| - if (r.ok) return await r.json(); |
| + const r = await fetchServiceGET(`/api/unfurl?url=${encodeURIComponent(url)}`); |
| + if (r && r.ok) return await r.json(); |
| } catch {} |
| } |
| try { |