amadan
geoip spec rev 1: platform GeoIP headers, mirrored database, feed, endpoint (#241)
Human
Prompt
Can we pick up discussion 241 and make a plan to implement?
From this commit's own message. The prompt ledger has no entries for it — which is what a squashed, rebased or amended commit looks like, and what every commit looks like in a repo without a ledger.
Summary
Design for exposing the platform's GeoIP lookup to apps, as approved in conversation, plus the Astra review brief.
2 files changed,
+625
−0
docs/superpowers/specs/2026-10-06-platform-geoip-design.md+559 −0docs/superpowers/specs/CODEX-REVIEW-BRIEF-GEOIP.md+66 −0
CI passed — run details
diff --git a/docs/superpowers/specs/2026-10-06-platform-geoip-design.md b/docs/superpowers/specs/2026-10-06-platform-geoip-design.md| new file mode 100644 |
| index 000000000..78091617f |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-06-platform-geoip-design.md |
| @@ -0,0 +1,559 @@ |
| +# Platform GeoIP for apps: headers, a mirrored database, a lookup endpoint |
| + |
| +**Date:** 2026-10-06 |
| +**Revision:** 1 |
| +**Discussion:** amadan carlos/platform #241 |
| +**Branch:** `geoip-for-apps` |
| + |
| +Rev 1 records the design Paul approved in conversation on 2026-10-06. It has |
| +not been reviewed by a second model yet. |
| + |
| +## 0. Rulings |
| + |
| +Paul's answers during brainstorming. Do not reopen them without him. |
| + |
| +- **R1. The database reaches the edges from the bucket.** The operator (or |
| + the feed, R6) publishes once. Every edge mirrors it and swaps its reader |
| + in place, with no restart. This replaces #241's "atomic rename plus |
| + restart" rule for the edge. `CONSOLE_GEOIP_DB` keeps that rule as an |
| + override (§11.3). |
| +- **R2. Four headers:** country, subdivision, coordinates and source (§7). |
| +- **R3. Tunnel and fleet boxes trust the hub.** The hub computes the |
| + headers. The remote edge keeps them only on requests that arrived over |
| + the authenticated tunnel (§8.3). Remote boxes hold no database. |
| +- **R4. No per-app switch.** Headers go to every app whenever a database |
| + is loaded. A switch can ride the telemetry settings pipeline later if |
| + someone asks for one. |
| +- **R5. One spec, three plans:** headers first, then the feed, then the |
| + endpoint (§16). Analytics countries ship in plan 1. |
| +- **R6. The database updates itself.** The console fetches new editions of |
| + DB-IP Lite or GeoLite2 and publishes them through the same path as a |
| + manual publish (§10). |
| + |
| +## 1. Why, and what this is not |
| + |
| +The nearest-region work gave the console a GeoIP reader (`internal/geo`, |
| +`CONSOLE_GEOIP_DB`). It serves one caller: guessing a new app's region. |
| +Once the platform carries the database, every app should be able to ask |
| +"roughly where is this visitor" without shipping its own `.mmdb`, handling |
| +attribution, or tracking updates. |
| + |
| +Goals: |
| + |
| +- Every proxied request to an app carries the visitor's country, and where |
| + the database has them, subdivision and coordinates. |
| +- Apps can look up an arbitrary IP (batch jobs, logs, webhooks). |
| +- The platform's own analytics fill the `countries` map the analytics spec |
| + reserved for this (`2026-09-07-errors-and-analytics-design.md`). |
| +- The operator publishes or configures a feed once per deployment, not once |
| + per box. |
| + |
| +Not goals: |
| + |
| +- **Accuracy beyond the database.** The headers say what the database says. |
| +- **Header injection for traffic that never crosses the edge.** Scheduler |
| + calls go straight to the socket. Sidecars reach their instance through |
| + loopback. |
| +- **ASN, ISP, timezone or city-name headers.** Additive later if asked for. |
| +- **A second reader.** Everything builds on `internal/geo`. |
| + |
| +## 2. Overview |
| + |
| +``` |
| +operator ──carlos system geoip publish──┐ |
| + ▼ |
| +feed (console, daily) ──────────► console publish path |
| + │ verify, upload blob, flip pointer |
| + ▼ |
| + bucket: control/geoip/current.json → control/geoip/<sha256>.mmdb |
| + │ |
| + every edge polls the pointer (10 min) |
| + ▼ |
| + /data/carlos/geoip/current.mmdb → geo.Swappable (hot swap) |
| + │ |
| + ServeHTTP: strip X-Carlos-Geo-*, look up clientIP(r), set headers |
| + │ │ |
| + ▼ ▼ |
| + proxied request to the app analytics Obs.Country |
| + → Day.Countries |
| +console: reads the same mirrored file → GET /api/cli/apps/{a}/{app}/geo?ip= |
| + carlos geo <ip> |
| +``` |
| + |
| +## 3. Bucket records |
| + |
| +The console is the **only writer** of `control/geoip/`. Edges only read it. |
| + |
| +**The blob** is `control/geoip/<sha256>.mmdb`. Its name is the hex SHA-256 |
| +of its bytes, so a blob never changes after it is written. |
| + |
| +**The pointer** is `control/geoip/current.json`: |
| + |
| +```json |
| +{ |
| + "v": 1, |
| + "sha256": "9f2c…", |
| + "key": "control/geoip/9f2c….mmdb", |
| + "size": 131072000, |
| + "database_type": "DBIP-City-Lite", |
| + "build_epoch": 1790812800, |
| + "published": "2026-10-06T21:00:00Z", |
| + "origin": "feed:dbip-city-lite", |
| + "edition": "2026-10" |
| +} |
| +``` |
| + |
| +- `database_type` and `build_epoch` come from the file's own MMDB metadata. |
| +- `origin` is `manual` or `feed:<name>`. `edition` is the feed's own edition |
| + marker: the `YYYY-MM` for DB-IP, the `Last-Modified` value for MaxMind, |
| + and empty for a manual publish. |
| +- **No pointer means the feature is off.** An unknown `v` is treated as no |
| + pointer, logged once. |
| + |
| +**Publish order:** the blob first, then the pointer, written with `ifMatch` |
| +on the pointer's previous etag so two publishers can't interleave. The |
| +previous blob is deleted only after the new pointer is written. An edge |
| +that read the old pointer moments before can find its blob gone. It |
| +retries on the next poll and reads the new pointer then. |
| + |
| +## 4. Publishing |
| + |
| +### 4.1 The console's publish path |
| + |
| +`geoipPublish(ctx, src io.Reader, origin, edition string)`, used by both |
| +the CLI verb and the feed: |
| + |
| +1. Stream `src` to a temporary file under the console's data directory, |
| + hashing as it goes. Refuse anything over 1 GiB. |
| +2. Open it with `geo.OpenMMDB`, which verifies it. |
| +3. Refuse a database with no country data. Probe: the reader's metadata |
| + lists a `country` field in its first record, or a lookup of a fixed set |
| + of well-known public addresses yields a country for at least one of |
| + them. The plan settles which check, against the real files. |
| +4. Refuse a database whose search tree has fewer than half the nodes of |
| + the current one (`metadata.node_count`). This guards against a truncated |
| + or wrong-product file. A manual publish can override it with `-force`. |
| + The feed can't. |
| +5. Upload the blob, write the pointer, delete the old blob (§3). |
| +6. Delete the temporary file. |
| + |
| +### 4.2 Operator verbs |
| + |
| +Under `carlos system`, through the console API like `carlos system |
| +antivirus`, for system operators only (`CARLOS_SYSTEM_OPERATORS`): |
| + |
| +- `carlos system geoip publish <file.mmdb> [-force]` uploads the file to |
| + the console, which runs §4.1. The body is streamed, not buffered. |
| +- `carlos system geoip status` shows: |
| + - the pointer (type, edition, build date, age, origin); |
| + - the feed's configuration and its last result; |
| + - the sha each box reports having loaded (§5.4), so "has every edge |
| + picked it up" is one command. |
| +- `carlos system geoip off` deletes the pointer, then the blob. Every edge |
| + drops its reader within one poll. The feed (§10) does not republish |
| + while the pointer is absent and the feed is unset. If the feed is set, |
| + `off` warns that the feed will publish again tomorrow and says to unset |
| + `CONSOLE_GEOIP_FEED`. |
| + |
| +## 5. The edge mirror |
| + |
| +### 5.1 Where and who |
| + |
| +- The mirror runs inside `carlos agent`'s edge on every box that reads the |
| + deployment bucket directly. |
| +- **It does not run on boxes that use the tunnel client** (`CARLOS_TUNNEL_*` |
| + set). Those get their headers from the hub (§8.3). |
| +- Files live under `/data/carlos/geoip/`, which already sits inside the |
| + edge unit's `ReadWritePaths`. The directory is mode 0755 and |
| + `current.mmdb` is 0644, so the console (user `carlos-console`, on the |
| + console box) can read the same file (§11.3). The data isn't secret. |
| + |
| +### 5.2 The loop |
| + |
| +- **At boot:** if `current.mmdb` exists, open it straight away. Serving |
| + never waits on the bucket. The sha it was published under is kept in |
| + `current.sha256` next to it. |
| +- **Every 10 minutes**, with jitter, read the pointer: |
| + - **Same sha as loaded:** nothing. |
| + - **New sha:** |
| + 1. Stream the blob to `.incoming` and check its sha and size against |
| + the pointer. |
| + 2. Open it with `geo.OpenMMDB`, off the request path. |
| + 3. Rename it to `current.mmdb`, write `current.sha256`, and swap the |
| + reader (§5.3). |
| + - **No pointer:** swap in a nil reader, then delete the local files. |
| + - **The read fails** (network, permissions): keep what is loaded. Log |
| + once per hour while it keeps failing. |
| +- **A blob that fails its sha or verification** is logged once per sha and |
| + not retried until the pointer changes. |
| + |
| +The current `release.Store.Get` returns the whole body as `[]byte`. For a |
| +DB-IP City file (about 130 MB) that's too much to hold on a small box, so |
| +plan 1 adds an optional streaming read: |
| + |
| +```go |
| +// release.StreamGetter is implemented by stores that can stream a body. |
| +type StreamGetter interface { |
| + GetTo(ctx context.Context, key string, w io.Writer) (etag string, err error) |
| +} |
| +``` |
| + |
| +The S3 store and the directory store implement it. The mirror uses it when |
| +it's there and falls back to `Get` when it isn't. |
| + |
| +### 5.3 Hot swap |
| + |
| +`geo.Swappable` wraps the reader: |
| + |
| +```go |
| +type Swappable struct { |
| + mu sync.RWMutex |
| + cur *MMDB |
| +} |
| +func (s *Swappable) Lookup(ip net.IP) (Result, bool) // RLock for the lookup |
| +func (s *Swappable) Swap(next *MMDB) // Lock, replace, Unlock, then Close the old one |
| +``` |
| + |
| +The old mapping is closed only after the write lock has been taken and |
| +released. No lookup is ever in flight on an unmapped file. A lookup holds |
| +the read lock for a few microseconds, so a swap waits that long at most. |
| +The swap needs a `-race` test with concurrent lookups (§14). |
| + |
| +### 5.4 Reporting |
| + |
| +Each box's status heartbeat (`status/boxes/<label>.json`) gains: |
| + |
| +```json |
| +"geoip": {"sha256": "9f2c…", "type": "DBIP-City-Lite", "error": ""} |
| +``` |
| + |
| +- The block is omitted entirely on tunnel boxes. |
| +- `carlos system geoip status` reads it (§4.2). |
| + |
| +## 6. `internal/geo` changes |
| + |
| +`Locate` stays as it is for the console's region guess. Lookups that need |
| +more get a new method: |
| + |
| +```go |
| +type Result struct { |
| + Country string // ISO 3166-1 alpha-2; country, else registered country |
| + Subdivision string // ISO 3166-2 like "GB-SCT"; empty unless the database has an ISO code |
| + Point Point |
| + HasPoint bool // true only for a real location record, never a centroid |
| +} |
| +func (m *MMDB) Lookup(ip net.IP) (Result, bool) |
| +func (m *MMDB) Source() string // metadata.DatabaseType, reduced to [A-Za-z0-9._-] |
| +``` |
| + |
| +- `Lookup` applies the same `Locatable` filter, `To4` normalisation, |
| + `onTheGlobe` check and fixed-category error log as `Locate`. |
| +- A malformed location makes the whole record a miss, as it does for |
| + `Locate`. A record wrong in one field isn't trusted on the next. |
| +- `ok` is false when no country can be found. |
| +- `Subdivision` is `country + "-" + subdivisions[0].iso_code`, only when |
| + both are present and the code is 1 to 3 `[A-Z0-9]` characters. |
| +- **DB-IP Lite has subdivision names but no ISO codes**, so with DB-IP the |
| + subdivision is always empty. With GeoLite2 City it is usually filled. |
| +- `Locate` is reimplemented on top of `Lookup` so the two can't drift. A |
| + record with no point still falls back to the country's representative |
| + point for `Locate`, as today. |
| + |
| +## 7. The header contract |
| + |
| +| Header | Value | Present when | |
| +|---|---|---| |
| +| `X-Carlos-Geo-Country` | ISO 3166-1 alpha-2, upper case | a country was found | |
| +| `X-Carlos-Geo-Subdivision` | ISO 3166-2, e.g. `GB-SCT` | the database has an ISO subdivision code (GeoLite2, not DB-IP Lite) | |
| +| `X-Carlos-Geo-Lat` | decimal degrees, 2 places | the record has a real location | |
| +| `X-Carlos-Geo-Lon` | decimal degrees, 2 places | the same | |
| +| `X-Carlos-Geo-Source` | the database type, e.g. `DBIP-City-Lite`, `GeoLite2-City` | any geo header is present | |
| + |
| +- All five are absent on a miss, on an address that isn't locatable, or |
| + when no database is loaded. The platform never guesses. |
| +- Two decimal places is about 1 km, roughly the precision of these |
| + databases anyway. The edge rounds before setting the header. |
| +- Apps can already see the client IP in `X-Forwarded-For`, so these headers |
| + tell an app nothing it couldn't work out for itself. |
| +- Apps that vary a cached response on a geo header must send `Vary` on it. |
| + The edge cache honours `Vary` with at most 8 variants per URL |
| + (`internal/edge/cache.go`). Country-level variation fits that budget. |
| + Coordinate-level variation does not, and the docs say so. |
| + |
| +## 8. Trust: who may set the headers |
| + |
| +### 8.1 The strip |
| + |
| +`ServeHTTP` deletes every `X-Carlos-Geo-*` header from the inbound request |
| +on **every** request: every route, every proxy type, before any branch. |
| +The one exception is §8.3. A test sends forged headers through each of the |
| +six proxy types (unix, tcp, multi, tunnel, fleet, backhaul) and asserts the |
| +app never sees them. |
| + |
| +### 8.2 Setting them |
| + |
| +After the strip, the edge looks up `clientIP(r)` and sets the headers. This |
| +happens after `acceptHop`, the route lookup and the gate, and **before** |
| +the upgrade branch and the cache lookup, so WebSocket upgrades carry them |
| +and `Vary` matching sees the right values. The result is also put on the |
| +request context so analytics reuses it (§9) instead of looking up twice. |
| + |
| +| Path | What happens | |
| +|---|---| |
| +| Direct request | `clientIP` is `RemoteAddr`. Look up. | |
| +| Verified backhaul hop (home edge) | `clientIP` is the last XFF element, trusted only under `hopVerifiedKey`. The home edge strips and recomputes. It doesn't trust the sending edge's copy. | |
| +| Backhaul sender | Strips like any edge, then forwards. It doesn't compute, because the home edge will. | |
| +| Unverified hop | Treated as a direct request from the sender's address, as today. | |
| +| Door (edge-local socket) | **No geo headers.** The door replaces `RemoteAddr` with the box's own public address, so a lookup would locate the box. | |
| +| Sidecar through loopback | The address isn't locatable, so there are no headers. | |
| +| Static sites served by the edge | No proxied request, so nothing to set. | |
| + |
| +### 8.3 Tunnel and fleet boxes |
| + |
| +The hub edge strips and sets the headers like any edge, then `tunnelProxy` |
| +or `fleetProxy` forwards them. On the remote box: |
| + |
| +- `tunnelclient` serves the edge over the inverted HTTP/2 connection. |
| + Plan 1 wraps that handler so every request it serves carries a new |
| + context mark, `tunnelKey{}`. Nothing else may set that mark, the same |
| + property `hopVerifiedKey` rests on. |
| +- The remote edge's strip skips requests that carry `tunnelKey`, and it |
| + doesn't look up for them either (its `RemoteAddr` is the hub, which would |
| + locate the hub). The hub's headers reach the app unchanged. |
| +- A remote box has no public listener for app traffic. If an operator ever |
| + configures one, requests through it don't carry `tunnelKey` and get |
| + stripped, so a forged header still can't get through. |
| + |
| +A test drives a request through a real `tunnelclient` connection and checks |
| +that the hub's headers arrive. Another test sends forged headers to the |
| +remote edge's own listener and checks they are stripped. |
| + |
| +### 8.4 Mixed versions during a roll |
| + |
| +An edge still on an old binary doesn't strip `X-Carlos-Geo-*`. That is |
| +true today for every edge. The contract is not real until every edge runs |
| +the new binary. So the rollout order (§15) is: roll every box, **then** |
| +publish a database, **then** publish the app docs. |
| + |
| +## 9. Analytics countries |
| + |
| +- `analytics.Obs` gains `Country string`. `observeAnalytics` reads it from |
| + the request context (§8.2) and doesn't look up again. |
| +- `Delta` and the flush fold `Country` into `Day.Countries`. An empty |
| + country counts as `"_unknown"`. The map is bounded like the other |
| + summaries: there are fewer than 260 ISO codes, so in practice it never |
| + hits the `_other` overflow, but the bound is enforced anyway. |
| +- The console's analytics page (`app_analytics.html`) gains a Countries |
| + table: the top 10, then "other". |
| +- **The page shows the attribution its database requires** (§13), chosen |
| + from the pointer's `database_type`. The new strings go through |
| + copy-review before they're written. |
| +- Days before plan 1 rolls have an empty map. The table says "no country |
| + data for this period" rather than showing nothing. |
| + |
| +## 10. The feed (plan 2) |
| + |
| +The console fetches new editions itself. It is the one control-plane |
| +process per deployment, so there is only ever one fetcher. |
| + |
| +### 10.1 Configuration (console env) |
| + |
| +| Variable | Meaning | |
| +|---|---| |
| +| `CONSOLE_GEOIP_FEED` | `dbip-city-lite`, `dbip-country-lite`, `geolite2-city`, `geolite2-country`; unset means no feed | |
| +| `CONSOLE_GEOIP_MAXMIND_ACCOUNT` | MaxMind account ID, for `geolite2-*` | |
| +| `CONSOLE_GEOIP_MAXMIND_KEY` | MaxMind license key, for `geolite2-*` | |
| + |
| +An unknown feed name, or a GeoLite2 feed with no credentials, is logged |
| +once at startup. The feed stays off and the console still starts. |
| + |
| +### 10.2 Checking for a new edition |
| + |
| +The console checks once a day, at a jittered time, and once 5 minutes after |
| +it starts. |
| + |
| +- **DB-IP:** the edition is `YYYY-MM`, from |
| + `https://download.db-ip.com/free/dbip-<city|country>-lite-YYYY-MM.mmdb.gz`. |
| + If the pointer's `edition` isn't this month, it tries this month's URL. A |
| + 404 just means it isn't out yet, so it tries again tomorrow. When nothing |
| + has been published yet, it tries this month and then last month. |
| +- **MaxMind:** it HEADs |
| + `https://download.maxmind.com/geoip/databases/GeoLite2-<City|Country>/download?suffix=tar.gz` |
| + with Basic auth and compares `Last-Modified` with the pointer's |
| + `edition`. MaxMind documents HEAD as not counting against the download |
| + limit. When the edition differs, it fetches `suffix=tar.gz.sha256` and |
| + then the archive. |
| + |
| +### 10.3 Integrity and safety |
| + |
| +- The download streams to disk and is decompressed: `.gz` for DB-IP, the |
| + single `.mmdb` member of the `.tar.gz` for MaxMind. Decompressed output |
| + is capped at 1 GiB, so a decompression bomb can't fill the disk. |
| +- MaxMind's archive is checked against its published SHA-256. DB-IP |
| + publishes no checksum that the console can fetch, so it relies on TLS, |
| + `Verify` and §4.1's checks. |
| +- The feed publishes only when the new database's `build_epoch` is |
| + **newer** than the current pointer's and its `database_type` **matches** |
| + the current one. A feed can't replace a City database with a Country one |
| + or roll back to an older edition. The one exception is when the current |
| + pointer is a manual publish of a different type: the feed logs that |
| + it's replacing it and goes ahead, because configuring a feed is the |
| + operator saying which database they want. |
| +- If the pointer is absent and the feed is set, the feed publishes. That |
| + is how the first publish happens on a fresh deployment. |
| +- On any failure the current database stays. Each edition's failure is |
| + logged once. |
| + |
| +### 10.4 Staleness |
| + |
| +When the pointer's `build_epoch` is more than 45 days old, the console |
| +raises a deployment-level alert, `geoip_stale`, through its existing alert |
| +watcher (`internal/console/alertwatch.go`). This happens whether or not a |
| +feed is set. The geo data keeps being served. Email to the grant |
| +recipients works. `CARLOS_ALERT_WEBHOOK` is still unset on the flagship |
| +(#192), so the webhook leg delivers nowhere until that is fixed. |
| + |
| +## 11. The lookup endpoint and CLI (plan 3) |
| + |
| +### 11.1 Endpoint |
| + |
| +`GET /api/cli/apps/{account}/{app}/geo?ip=<addr>`, registered through |
| +`cliRoute`: |
| + |
| +- **Allowed:** the instance token (`svcrole.Instance`), and the member and |
| + app tokens that can read the app's analytics. |
| +- **Rate limit:** the existing per-caller limiter at 60 a minute. The |
| + intended use is occasional lookups, not bulk enrichment; a bulk form can |
| + come later if anyone needs it. |
| +- **200** `{"found": true, "country": "IE", "subdivision": "", "lat": 53.34, "lon": -6.27, "source": "DBIP-City-Lite", "attribution": "IP Geolocation by DB-IP", "attribution_url": "https://db-ip.com"}`. |
| + The fields follow §7: absent values are omitted, and the coordinates |
| + are rounded the same way. |
| +- **200** `{"found": false}` for an address that isn't locatable or isn't |
| + in the database. |
| +- **400** for a missing or malformed `ip`. |
| +- **503** `geo unavailable` when no database is loaded. |
| +- The console never logs the queried IP. |
| + |
| +### 11.2 CLI |
| + |
| +`carlos geo <ip> [--app <app>]` calls the endpoint and prints the fields |
| +and the attribution. `--json` prints the response as is. With no app |
| +context it uses the current app, like the other app verbs. |
| + |
| +### 11.3 The console's reader |
| + |
| +The console reads `/data/carlos/geoip/current.mmdb`, the file the console |
| +box's own edge mirrors. Every minute it checks the file's inode and swaps |
| +the reader (`geo.Swappable`) when the inode changes. Region guesses and the |
| +endpoint share that one reader. |
| + |
| +`CONSOLE_GEOIP_DB`, when set, overrides the mirrored path and keeps its |
| +current rules: it's opened once, and replacing it means a rename plus a |
| +restart. That keeps existing deployments working unchanged. |
| + |
| +## 12. Logging |
| + |
| +Every message is a fixed category and **never includes an IP address**. |
| + |
| +| Where | Message | Rate | |
| +|---|---|---| |
| +| edge mirror | `edge: geoip database <sha8> (<type>, built <date>) loaded` | per swap | |
| +| edge mirror | `edge: geoip turned off; no database published` | per transition | |
| +| edge mirror | `edge: geoip blob <sha8> rejected: <category>` | once per sha | |
| +| edge mirror | `edge: geoip pointer unreadable; keeping the loaded database` | hourly while failing | |
| +| geo | `geo: a GeoIP lookup failed and was treated as a miss` | once a minute, process-wide (existing) | |
| +| console | `console: geoip published <sha8> (<type>, <origin>)` | per publish | |
| +| console feed | `console: geoip feed <name>: edition <e> rejected: <category>` | once per edition | |
| +| console feed | `console: geoip feed misconfigured: <category>` | at startup | |
| + |
| +## 13. Licensing and docs |
| + |
| +- **DB-IP Lite** is licensed CC BY 4.0. Pages that show or use its data |
| + need `IP Geolocation by DB-IP` with a link to `https://db-ip.com`. |
| +- **GeoLite2** is covered by MaxMind's EULA. It requires attribution |
| + (`This product includes GeoLite2 data created by MaxMind, available from |
| + https://www.maxmind.com`) and keeping the data current, which the feed |
| + does. |
| +- **The operator owns compliance.** The operator docs say that plainly, |
| + next to the publish verb and the feed settings. |
| +- **The app docs** list the headers, explain that they may be absent and |
| + why, cover `Vary` and its limit, and spell out the duty: an app that |
| + shows visitors anything derived from these headers or the endpoint must |
| + carry the attribution `X-Carlos-Geo-Source` names. The endpoint returns |
| + the attribution text so an app can render it. |
| +- **The platform's own pages** that use the data carry the attribution: |
| + the analytics Countries table, and the new-app region guess if it ever |
| + shows the visitor where it thinks they are. |
| + |
| +## 14. Testing |
| + |
| +- **`internal/geo`:** |
| + - `Lookup` against the MaxMind fixtures already in `testdata` |
| + (country, registered-country fallback, subdivision codes from |
| + `GeoIP2-City-Test.mmdb`, no point from a Country database); |
| + - malformed locations; |
| + - `Locate` returning exactly what it did before (the existing tests |
| + stay); |
| + - `Source` sanitising. |
| +- **Swap:** `-race`, with 32 goroutines looking up while 100 swaps happen, |
| + and a `Close` spy proving no close happens while a read lock is held. |
| +- **Mirror**, against the in-memory store: |
| + - first publish, a new sha, the pointer removed, the pointer |
| + unreadable; |
| + - a blob with a sha mismatch, a blob that fails `Verify`; |
| + - boot with a local file and no bucket; |
| + - the old blob deleted between reading the pointer and fetching the |
| + blob. |
| +- **Edge:** |
| + - the strip test across all six proxy types; |
| + - setting on a direct request; |
| + - recomputing on a verified hop, never trusting a sender's copy; |
| + - none on the door, none for loopback; |
| + - the tunnel mark passing the hub's headers; |
| + - a forged header on a tunnel box's own listener being stripped; |
| + - headers present on a WebSocket upgrade; |
| + - the cache's `Vary` variant selected by country; |
| + - analytics `Countries` filled from the same lookup. |
| +- **Publish path:** each refusal in §4.1, the `ifMatch` race, the |
| + old-blob delete. |
| +- **Feed**, against an `httptest` server standing in for DB-IP and MaxMind: |
| + - a new edition; |
| + - a 404 before the month's file appears; |
| + - an older `build_epoch`, a type mismatch; |
| + - a bad SHA-256, a decompression bomb; |
| + - HEAD-only when `Last-Modified` hasn't changed; |
| + - the staleness alert. |
| +- **Endpoint:** each status code, role refusals, the rate limit, and that |
| + no IP ever reaches the log. |
| + |
| +## 15. Rollout |
| + |
| +1. Merge plan 1 and roll every box. There is no pointer yet, so nothing |
| + changes. Strips happen, but there is nothing to strip. |
| +2. Confirm with `carlos system geoip status` that every box reports a |
| + mirror with no database. |
| +3. Publish a database by hand (plan 1), or wait for plan 2's feed. Confirm |
| + every box reports the sha. |
| +4. Check a request to a test app carries the headers. Check a forged header |
| + from outside is replaced. |
| +5. Publish the app docs. |
| + |
| +Plans 2 and 3 roll the console only. The feed does nothing until |
| +`CONSOLE_GEOIP_FEED` is set. |
| + |
| +## 16. Plans |
| + |
| +| Plan | Contents | |
| +|---|---| |
| +| 1 | `geo.Lookup`, `geo.Swappable`, `release.StreamGetter`, the console publish path and `carlos system geoip publish|status|off`, the edge mirror and box status, strip and set, the tunnel mark, analytics `Countries` and the console table, operator and app docs | |
| +| 2 | The feed: DB-IP and MaxMind sources, integrity checks, monotonic publish, the `geoip_stale` alert | |
| +| 3 | The console reader on the mirrored file, the endpoint, `carlos geo` | |
| + |
| +## 17. Open items for the plans |
| + |
| +- **The country-data probe in §4.1.3** (metadata versus known addresses) |
| + is decided against a real DB-IP Lite and GeoLite2 file in plan 1, task 1. |
| + That task also confirms that DB-IP Lite has no subdivision ISO codes. |
| +- **DB-IP's site says nothing on automated downloads.** Plan 2 fetches at |
| + most once a day, and only when a new month is due. If DB-IP objects, the |
| + feed falls back to manual publishing with no code change. |
diff --git a/docs/superpowers/specs/CODEX-REVIEW-BRIEF-GEOIP.md b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-GEOIP.md| new file mode 100644 |
| index 000000000..cfd3e3ee8 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-GEOIP.md |
| @@ -0,0 +1,66 @@ |
| +# Review brief: platform GeoIP for apps |
| + |
| +You are GPT-6 reviewing a design spec independently of its author. |
| +Your output is a findings report, not an edit. Do not modify any file. |
| + |
| +## What to read |
| + |
| +1. The spec under review: |
| + `docs/superpowers/specs/2026-10-06-platform-geoip-design.md`. |
| + §0's rulings are Paul's and fixed. Report a mechanism that fails one; |
| + do not re-argue one. |
| +2. The originating request: amadan discussion carlos/platform #241, quoted |
| + in the spec's §1. The nearest-region spec it builds on: |
| + `docs/superpowers/specs/2026-10-01-nearest-region-default-design.md` §6. |
| +3. The code it fits into, in this worktree: |
| + - `internal/geo/` (`mmdb.go`, `point.go`, `tz.go`), `cmd/console/main.go` |
| + (`openGeoIP`); |
| + - `internal/edge/edge.go` (`ServeHTTP`, the six proxies and their |
| + `Rewrite` functions, `observeAnalytics` defer), `internal/edge/homemap.go` |
| + (`acceptHop`, `setForwarded`, `stripForwardedHeaders`), |
| + `internal/edge/analytics.go` (`clientIP`, `hopVerified`), |
| + `internal/edge/door.go`, `internal/edge/cache.go`, |
| + `internal/edge/certsync.go` (the precedent for mirroring from the bucket), |
| + `internal/edge/purge_header.go`; |
| + - `internal/tunnelclient/`, `internal/host/tunnel_client.go`, |
| + `internal/host/agent.go`, `internal/host/units/carlos-edge.service`; |
| + - `internal/release/store.go` (`Store`), the S3 store; |
| + - `internal/analytics/` (`Day`, `Delta`, `Obs`, flush and merge); |
| + - `internal/console/authcli.go` (`cliRoute`, roles, rate limiting), |
| + `internal/console/alertwatch.go`, `system.go` (the `carlos system |
| + antivirus` verbs as the precedent for operator verbs); |
| + - `AGENTS.md`. |
| + |
| +## What matters most |
| + |
| +- **Forgery.** Find any path by which a client-chosen value reaches an app |
| + in an `X-Carlos-Geo-*` header: any proxy, hop, tunnel, door, upgrade or |
| + cache path the strip in §8 misses, or a way to get `tunnelKey` onto a |
| + request that did not arrive over the tunnel. |
| +- **Wrong location.** Any path where the edge locates the wrong address |
| + (the box itself, the hub, a sender edge, a proxy) and sets headers that |
| + look authoritative. |
| +- **The hot swap (§5.3).** Can a lookup ever run against a closed mapping? |
| + Any deadlock or starvation under load? |
| +- **The mirror and publish protocol (§3–§5).** Races between publishers, |
| + between a publish and an edge poll, a deleted blob, a partially written |
| + file, disk filling, a box restarting mid-swap, the console box reading a |
| + file the edge is replacing. |
| +- **The feed (§10).** Supply-chain risks in fetching and publishing a file |
| + every edge maps into memory; decompression limits; the monotonic rule; |
| + credential handling; behaviour when DB-IP or MaxMind change URLs. |
| +- **Cache interaction (§7, §8.2).** Does setting headers before the cache |
| + lookup interact correctly with `Vary`, the 8-variant limit and purge? |
| +- **Licensing (§13).** Is anything stated wrongly about DB-IP Lite's |
| + CC BY 4.0 terms or MaxMind's GeoLite2 EULA? |
| +- **Claims about the code.** Every `file:line` and every "already exists" |
| + claim: check it. Report each one that is wrong. |
| +- **Completeness for planning.** Anything an implementer would have to |
| + invent because the spec is silent. |
| + |
| +## Output |
| + |
| +A numbered list of findings, most severe first, each with: severity |
| +(P1 = wrong or unsafe as written, P2 = gap an implementer would get wrong, |
| +P3 = clarity), the spec section, the code evidence with file:line, and a |
| +concrete fix. End with a one-line verdict: accept, or revise. |