amadan
spec: edge request protection design, draft for builder check
Probe and secret blocking before wake, OWASP CRS PL1 via Coraza in watch-then-block, tenant rules per app (file and console union), request records, under-attack proof of work. Rate limits defer to #358.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+542
−0
docs/superpowers/specs/2026-10-24-edge-request-protection-design.md+542 −0
CI cancelled — superseded by a newer push — run details
CI log
=== gate === .amadan/test-gate.sh === launchctx (1 s) ok amadan.net/carlos/platform.git/launchctx 0.013s ? amadan.net/carlos/platform.git/launchctx/controlname [no test files] ? amadan.net/carlos/platform.git/launchctx/wireid [no test files] === vet (18 s) === gofmt (1 s) gofmt -l .: clean === rest (171 s) ? github.com/carlosframework/platform/cmd/changelog [no test files] ok github.com/carlosframework/platform/cmd/console 3.782s ok github.com/carlosframework/platform/cmd/moveprobe 11.838s ok github.com/carlosframework/platform/cmd/zddprobe 0.766s ok github.com/carlosframework/platform/internal/activator 58.074s ok github.com/carlosframework/platform/internal/agentsock 0.060s ok github.com/carlosframework/platform/internal/alerting 0.008s ok github.com/carlosframework/platform/internal/analytics 0.031s ok github.com/carlosframework/platform/internal/appconfig 2.871s ok github.com/carlosframework/platform/internal/apperr 0.150s ok github.com/carlosframework/platform/internal/applog 0.217s ok github.com/carlosframework/platform/internal/awsinv 0.004s ok github.com/carlosframework/platform/internal/boxchoice 0.006s ok github.com/carlosframework/platform/internal/boxid 0.868s ? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files] ? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files] ok github.com/carlosframework/platform/internal/boxvolumes 0.014s ok github.com/carlosframework/platform/internal/caps 0.016s ok github.com/carlosframework/platform/internal/certwarm 0.005s ok github.com/carlosframework/platform/internal/cgroup 0.785s ok github.com/carlosframework/platform/internal/changelog 0.516s ok github.com/carlosframework/platform/internal/claimactivation 1.346s ok github.com/carlosframework/platform/internal/clamd 0.075s ? github.com/carlosframework/platform/internal/cliflag [no test files] ? github.com/carlosframework/platform/internal/controlname [no test files] ? github.com/carlosframework/platform/internal/controlsock [no test files] ok github.com/carlosframework/platform/internal/dbdoor 8.263s ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.013s ok github.com/carlosframework/platform/internal/dbquery 0.013s ok github.com/carlosframework/platform/internal/dbreads 2.162s ? github.com/carlosframework/platform/internal/dbreads/dbreadstest [no test files] ok github.com/carlosframework/platform/internal/dbverify 0.005s ok github.com/carlosframework/platform/internal/dnsrec 0.006s ok github.com/carlosframework/platform/internal/docsite 0.014s ok github.com/carlosframework/platform/internal/domains 2.809s ok github.com/carlosframework/platform/internal/doorkey 0.006s ok github.com/carlosframework/platform/internal/economics 0.007s ok github.com/carlosframework/platform/internal/edge 45.669s ? github.com/carlosframework/platform/internal/edgeproof [no test files] ok github.com/carlosframework/platform/internal/edgesfile 0.008s ok github.com/carlosframework/platform/internal/fleetapi 0.164s ok github.com/carlosframework/platform/internal/gate 1.727s ok github.com/carlosframework/platform/internal/genselect 0.024s ok github.com/carlosframework/platform/internal/geo 0.023s ok github.com/carlosframework/platform/internal/geoip 0.036s ok github.com/carlosframework/platform/internal/historyreadback 0.292s ok github.com/carlosframework/platform/internal/host 142.050s ok github.com/carlosframework/platform/internal/hostactors 6.281s ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.016s ok github.com/carlosframework/platform/internal/hostbox 3.094s ok github.com/carlosframework/platform/internal/hostclaim 0.779s ok github.com/carlosframework/platform/internal/hostenv 0.016s ok github.com/carlosframework/platform/internal/hostevidence 0.016s ok github.com/carlosframework/platform/internal/hostexec 5.401s ok github.com/carlosframework/platform/internal/hostfence 0.040s ok github.com/carlosframework/platform/internal/hostfiles 10.386s ok github.com/carlosframework/platform/internal/hostfloor 0.017s ok github.com/carlosframework/platform/internal/hostguard 1.076s ok github.com/carlosframework/platform/internal/hostops 2.005s ? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files] ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.016s ok github.com/carlosframework/platform/internal/hostquarantine 0.016s ok github.com/carlosframework/platform/internal/hostrelocate 0.622s ok github.com/carlosframework/platform/internal/hostresolve 0.229s ok github.com/carlosframework/platform/internal/hostwrite 0.018s ok github.com/carlosframework/platform/internal/infraview 0.017s ok github.com/carlosframework/platform/internal/instancectl 5.254s ok github.com/carlosframework/platform/internal/instanceexec 2.825s ok github.com/carlosframework/platform/internal/instancefs 0.022s ok github.com/carlosframework/platform/internal/instancekey 0.018s ok github.com/carlosframework/platform/internal/instanceupdates 0.025s ok github.com/carlosframework/platform/internal/layout 1.790s ok github.com/carlosframework/platform/internal/lbhealth 0.015s ok github.com/carlosframework/platform/internal/lease 10.353s ok github.com/carlosframework/platform/internal/lease/leasetest 0.612s ok github.com/carlosframework/platform/internal/ledger 0.027s ok github.com/carlosframework/platform/internal/mail 0.231s ok github.com/carlosframework/platform/internal/maintenance 0.094s ok github.com/carlosframework/platform/internal/maintenanceexec 5.600s ok github.com/carlosframework/platform/internal/maintenanceguard 0.107s ok github.com/carlosframework/platform/internal/metrics 0.009s ok github.com/carlosframework/platform/internal/moves 22.539s ok github.com/carlosframework/platform/internal/moves/archive 0.956s ok github.com/carlosframework/platform/internal/objstore 0.281s ok github.com/carlosframework/platform/internal/ops 0.239s ok github.com/carlosframework/platform/internal/park 0.300s ok github.com/carlosframework/platform/internal/plans 0.012s ok github.com/carlosframework/platform/internal/privatefile 0.011s ok github.com/carlosframework/platform/internal/provision 24.575s ok github.com/carlosframework/platform/internal/publicrelease 0.009s ? github.com/carlosframework/platform/internal/regionname [no test files] ok github.com/carlosframework/platform/internal/registry 5.594s ok github.com/carlosframework/platform/internal/relay 7.220s ok github.com/carlosframework/platform/internal/release 26.455s ok github.com/carlosframework/platform/internal/rollout 3.903s ok github.com/carlosframework/platform/internal/rolloutwords 0.007s ok github.com/carlosframework/platform/internal/rootaws 0.009s ok github.com/carlosframework/platform/internal/rootparticipation 0.061s ok github.com/carlosframework/platform/internal/rootstate 0.026s ok github.com/carlosframework/platform/internal/runs 0.568s ok github.com/carlosframework/platform/internal/s3compat 0.005s ok github.com/carlosframework/platform/internal/safefile 0.004s ok github.com/carlosframework/platform/internal/schedule 2.674s ok github.com/carlosframework/platform/internal/seeds 0.014s ok github.com/carlosframework/platform/internal/sim/engines 2.598s ok github.com/carlosframework/platform/internal/sim/hostlife 11.460s ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.462s ok github.com/carlosframework/platform/internal/status 0.114s ok github.com/carlosframework/platform/internal/steering 0.008s ok github.com/carlosframework/platform/internal/stopevidence 0.316s ok github.com/carlosframework/platform/internal/svcrole 0.002s ok github.com/carlosframework/platform/internal/telemetry 0.006s ok github.com/carlosframework/platform/internal/termfence 0.005s ok github.com/carlosframework/platform/internal/tunnelclient 0.005s ok github.com/carlosframework/platform/internal/unixdial 0.002s ok github.com/carlosframework/platform/internal/updatewords 0.005s ok github.com/carlosframework/platform/internal/usage 0.005s ok github.com/carlosframework/platform/internal/version 0.004s ok github.com/carlosframework/platform/internal/vet 1.689s === internal_console@0 (81 s) ok github.com/carlosframework/platform/internal/console 79.203s === internal_console@1 (44 s) ok github.com/carlosframework/platform/internal/console 42.055s === internal_console@2 (42 s) ok github.com/carlosframework/platform/internal/console 40.048s === internal_console@3 (42 s) ok github.com/carlosframework/platform/internal/console 40.291s === internal_console@4 (43 s) ok github.com/carlosframework/platform/internal/console 40.985s === internal_console@5 (42 s) ok github.com/carlosframework/platform/internal/console 40.246s === internal_console@6 (38 s) ok github.com/carlosframework/platform/internal/console 36.116s === internal_console@7 (46 s) ok github.com/carlosframework/platform/internal/console 44.170s === root@0 (110 s) ok github.com/carlosframework/platform 108.137s === root@1 (84 s) ok github.com/carlosframework/platform 82.485s === root@2 (93 s) ok github.com/carlosframework/platform 91.104s === cross === --- shard 0 of 5 --- cross-compiling darwin/arm64 amadan: job cancelled — a newer push superseded it --- shard 1 of 5 --- cross-compiling darwin/amd64 --- shard 2 of 5 --- cross-compiling linux/arm64 amadan: job cancelled — a newer push superseded it --- shard 3 of 5 --- cross-compiling windows/amd64 --- shard 4 of 5 --- cross-compiling windows/arm64 amadan: job cancelled — a newer push superseded it
diff --git a/docs/superpowers/specs/2026-10-24-edge-request-protection-design.md b/docs/superpowers/specs/2026-10-24-edge-request-protection-design.md| new file mode 100644 |
| index 000000000..b21702aaa |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-24-edge-request-protection-design.md |
| @@ -0,0 +1,542 @@ |
| +# Edge request protection: probe blocking, attack rules and tenant rules |
| + |
| +**Date:** 2026-10-24 · **Status:** draft, awaiting Paul's read · **Requested |
| +by:** Paul, in conversation ("Should CARLOS platform have the equivalent of a |
| +WAF built in?", then "Can we get a more extensive database of well known probe |
| +paths? Can we do 80/20 on OWASP style stuff? Can we allow tenants to define |
| +their own rules per app?"). |
| + |
| +**Builds on:** the password gate's route-property path |
| +(`docs/superpowers/specs/2026-09-07-password-gate-design.md`), the edge rate |
| +limit (`docs/superpowers/specs/2026-10-24-edge-rate-limit-design.md` on branch |
| +`design-edge-rate-limit-358`, amadan #358, approved), the edge's trusted client |
| +address (`clientIP`, `internal/edge/analytics.go`), per-app log shipping |
| +(`internal/applog`, whose `StreamEdge` "per-request access lines (stage 2)" is |
| +declared and never emitted), the static serving class (`internal/edge/static.go`) |
| +and GeoIP (#241). |
| + |
| +Facts below about current code were checked on origin/main b612e710. |
| + |
| +## 1. What this is for |
| + |
| +Paul wants CARLOS to protect every app it hosts against the common attacks |
| +before carloku.com is advertised more widely. There have been many security |
| +incidents in the industry lately, and "every app sits behind sensible |
| +protection by default" is something the platform should be able to say. |
| + |
| +It is a platform feature. carloku.com gets it as one tenant among many, and a |
| +self-hosted platform gets it with no extra service. |
| + |
| +**The overriding priority, in Paul's words: a misfire is worse than a miss, |
| +but both hurt.** So the design aims for high-precision blocking, not timid |
| +watching: block by default only what is near-certain, earn every other default |
| +from real traffic, and make a misfire cheap to fix (a reference code on the |
| +block page, the reason visible to the developer, a one-line exception that |
| +takes effect in seconds). |
| + |
| +## 2. Decisions Paul made (2026-10-24, in conversation) |
| + |
| +1. **Not a full WAF; a platform-shaped layer.** No managed signature WAF in |
| + front of the boxes, no IP reputation lists. The edge gets protections that |
| + fit a shared, multi-tenant front door. |
| +2. **Both outcomes hurt; precision over timidity.** Near-certain rules block by |
| + default. Everything else runs in watch mode first and is promoted to |
| + blocking only when real traffic shows it does not misfire. |
| +3. **Cheap to fix.** Every block carries a reference code; the developer can see |
| + which rule fired and add an exception in one line, live within seconds. |
| +4. **Tenant rules are written by the app's developer**, typically reacting to a |
| + known attack that may be under way. They must **take effect within seconds, |
| + without a deploy**. |
| +5. **Rules can be written in a file and in the console: a union** (Paul: "Can |
| + we do both or either or a union?"). Each rule records where it came from; |
| + pushing the file never touches console rules (§6.2). |
| +6. **Serve a probe-looking path if the app really has it** (Paul: "if /wp-admin |
| + exists we should serve it"). Blocking a path the app would 404 anyway costs |
| + the visitor nothing; the design checks existence wherever it can (§5.3). |
| +7. **Except obvious security leaks: block and warn, with a hard override** |
| + (Paul: "block and warn for obvious security vulnerabilities that would |
| + require a hard override"). `/.env`, `/.git/` and friends are blocked even |
| + when the file exists, the developer is warned at deploy time, and serving |
| + one needs a deliberate override for one exact path (§5.2). |
| +8. **Rate limits are #358's**, already designed and approved: per instance, |
| + applying only while the instance is busy, plan-bounded on carloku.com. This |
| + design adds only a probe-specific limit (§5.5) and does not redesign them. |
| + (In conversation I offered "rate-limit" as a tenant rule action; that is |
| + withdrawn in favour of #358's per-instance path exceptions.) |
| + |
| +## 3. Evidence (recon, 2026-10-24) |
| + |
| +**CARLOS records nothing per request today.** Analytics keeps a path only for |
| +200 `text/html` page views; errors only 5xx; the usage meter only counts; the |
| +edge access stream is declared but never written; journald on the boxes holds |
| +TLS handshake and proxy errors only, and on the busiest box covers about 12 |
| +hours because replica-sync INFO lines fill it. So the evidence below came from |
| +the legacy (non-CARLOS) stack of a self-hosted deployment with more real |
| +traffic, which sits behind a managed cloud WAF: 7 days of CDN WAF logs (13.4M |
| +requests) and the access logs of a load balancer with no healthy targets (pure |
| +background scanning, 14.5k requests). |
| + |
| +**What attackers ask for** (about 55k probe requests in 7 days): |
| + |
| +| Target | Hits | Distinct paths | Leading paths | |
| +|---|---|---|---| |
| +| Secrets and dotfiles | 17.5k | 2,825 | `/.git/config`, `/.env` and dozens of `.env.*` variants, `/<dir>/.env` for ~40 dirs, `/.aws/credentials`, `/.docker/config.json`, `/.git-credentials`, `/docker-compose.yml`, `/@fs/.env` | |
| +| PHP scripts and webshells | 12.5k | 2,015 | `/admin.php`, `/info.php`, `/phpinfo.php`, `/_profiler/phpinfo`, long tail of shell names | |
| +| Backups and dumps | 11.0k | 1,528 | `/db.sql`, `*.php.save`, `*.zip`, nothing over ~40 each | |
| +| WordPress | 7.9k | 1,219 | `/wordpress/`, `/wp-json/` under many prefixes, `/wp-login.php`, `/xmlrpc.php`, plugin file managers | |
| +| Admin panels | ~4.0k | 210 | `/admin/login`, `/admin`, `/admin/settings` | |
| +| Smaller groups | 1.6k | | PHP frameworks (`/vendor/phpunit`, `/_ignition`, `/telescope`, ThinkPHP `invokefunction`), Java/Spring (`/actuator`, `.action`, `.do`, `.jsp`), phpMyAdmin, Node/Vite dev (`/@fs/`), VPN/Exchange, CGI/router/IoT (`/cgi-bin/luci`, `/setup.cgi`, `/HNAP1`, `/GponForm`), `/server-status`, cloud metadata | |
| + |
| +Newer targets: `/_debugbar`, `/vendor/composer/installed.json`, `/Dockerfile`, |
| +`/gcp-credentials.json`, `/firebase-adminsdk.json`, `POST /mcp`, `/dns-query`. |
| + |
| +**Attack payloads were rare:** ~830 path traversal / file inclusion, 175 |
| +server-side request forgery (cloud metadata URLs, `file:///proc/self/environ`), |
| +119 SQL injection (one source), ~50 PHP injection, 8 shell injection, 4 XSS, |
| +zero Log4Shell. The managed WAF let **every** SQL injection through (its common |
| +rule set has no SQL rules) and allowed 84 Vite `/@fs/` and `/proc/self/environ` |
| +requests. |
| + |
| +**Who sends it:** 55-65% of hostile traffic claims to be a browser, so |
| +user-agent rules catch little. One source sent 2,946 requests in a minute across |
| +~1,300 distinct probe paths; a limit on "not found" responses per client would |
| +stop that. |
| + |
| +**Probes cost CARLOS wakes.** Any request to a hibernating route wakes the |
| +instance unless the response cache answers it. In the same 7 days the flagship |
| +had 3,683 wakes (3,161 with no scheduled job within 90 s, across 96 hosts) and |
| +the self-hosted deployment 1,205 (1,196 unscheduled). They can't be attributed |
| +to bots without request records, but every probe on a known host costs a wake. |
| + |
| +**What misfired on real visitors** (the most important finding): |
| + |
| +1. **IP reputation lists**: ~22% of sampled reputation-list blocks were real |
| + visitors (browser fetch headers, same-site Referer, a mobile app) loading an |
| + embed widget, sign-in assets and event images. Roughly 4k real requests a |
| + week. |
| +2. **Bad-bot user-agent rule**: 65.5k blocks, nearly all a legitimate SEO |
| + crawler many site owners want. |
| +3. **Cookie and query-string XSS rules, URI-size rule**: fired on real widget |
| + loads, a native app's signed image URL (random base64 that looked like XSS) |
| + and a long signed image-proxy URL. |
| +4. **SQL words in real paths** a naive matcher would block: |
| + `…-vat-in-the-european-union`, `…display-a-selection-of-tickets`, |
| + `…update-the-widget-from-version-1`, `…how-do-i-delete-a-hub`, |
| + `…export-attendees-from-all-events`, `…embeddable-javascript-widget`. |
| +5. **"admin" in real paths**: `/assets/admin/application-<hash>.js` (380 hits, |
| + 281 sources), `favicon-admin-*.png`, a help article about the admin pane. |
| +6. **"wp-" and "pma"**: ~560 real hotlinked `/wp-content/uploads/…png` loads a |
| + week; a customer upload named `PMA_FULL_LOGO…` matched as phpMyAdmin. |
| +7. **Random base64** in signed URL tokens containing `eval`, `drop`, `union`, |
| + `exec`, `sleep` (176 requests a week). |
| +8. **URLs inside query parameters** (OAuth `redirect_uri=https://…`), ~600 a |
| + week: a naive SSRF rule would break sign-in. |
| +9. **`..`, quotes and backslashes** in real URLs: a cross-domain analytics |
| + parameter ending `..`, image filenames containing `..`, broken escaped links |
| + `/en/articles/\"https:/…\"`. |
| + |
| +**From a tenant app's own abuse protection** (read-only review of its source): |
| +it uses no user-agent checks, IP reputation, header-consistency scoring or |
| +fingerprinting, and its design notes reject them because an attacker passes by |
| +copying one accepted request. It relies on rate limits that slow rather than |
| +block (429 with Retry-After, penalties that grow then decay), IPv6 counted per |
| +/64, proof of work instead of CAPTCHAs, refusals tallied in memory and flushed |
| +in batches rather than written per request, and leaving ordinary page views and |
| +crawlers alone. Misfires it caught before and after launch: a "form filled in |
| +too fast" check refused decisive people; a honeypot field named `website` was |
| +filled by password managers; checkout steps that never received the spam-check |
| +token refused real buyers. Its rate limits were broken twice by trusting the |
| +wrong client address. The edge's `clientIP` already gets this right. |
| + |
| +## 4. Goals, non-goals, anti-goals |
| + |
| +**Goals** |
| + |
| +- Request records good enough to calibrate rules, explain any block from its |
| + reference code, and show a developer what a rule would have caught. |
| +- Hard blocks for obvious leaks, on by default for every app. |
| +- A maintained probe catalogue, blocked only where the app would not serve the |
| + path, before any wake. |
| +- A limit on probe bursts that never penalises a shared address's real users. |
| +- The OWASP core rule set's 80/20: its lowest-strictness rules, watch first, |
| + then blocking for the ones that earn it. |
| +- Tenant rules per app: written in a file or the console, live in seconds, |
| + undone as fast. |
| +- A per-app "under attack" switch using a proof-of-work challenge. |
| + |
| +**Non-goals (v1)** |
| + |
| +- Rate limiting in general (#358 owns it). |
| +- Volumetric floods and DDoS from many addresses (the cloud provider's network |
| + layer). |
| +- IP reputation lists, user-agent blocking, browser fingerprinting, CAPTCHAs. |
| +- Response inspection (data-leak detection in what the app sends back). |
| +- Tenant rules that match request bodies (v1 matches the request line, headers, |
| + client address and country; the OWASP rules do inspect bodies). |
| +- Automatic "under attack" triggering. |
| +- Bot management for good bots (verifying crawlers); good bots are simply not |
| + targeted. |
| + |
| +**Anti-goals: it fails if any of these happen, however well it works otherwise** |
| + |
| +- A real visitor is blocked and nobody can tell why: no reference code, or the |
| + developer cannot find the rule from it. |
| +- A developer cannot stop a misfire within a minute, without a deploy. |
| +- A blocked probe wakes a sleeping instance or reaches it at all. |
| +- A rule matches on a word inside a path (`union`, `admin`, `wp-`) rather than |
| + on path structure or parsed syntax. |
| +- One app's rule affects another app. |
| +- Request records store passwords, tokens, cookie values or full client |
| + addresses, or cost a storage write per request. |
| +- It adds latency a visitor would notice on an ordinary page view. |
| +- A self-hosted platform needs an AWS service, a plan catalog or anything |
| + carloku.com-specific to use it (provider-neutral core). |
| + |
| +## 5. Platform protections |
| + |
| +### 5.1 Tiers |
| + |
| +Every platform rule belongs to one tier: |
| + |
| +| Tier | Default | When the path exists in the app | Override | |
| +|---|---|---|---| |
| +| **Hard** (§5.2): obvious leaks and traversal | block | still blocked, and the developer is warned at deploy time | hard override, one exact path | |
| +| **Probe** (§5.3): technology-specific probe paths | block, when the app would not serve the path | served | ordinary allow rule | |
| +| **Attack** (§5.4): OWASP core rule set, lowest strictness | watch, then block per rule once calibrated | n/a | ordinary allow rule, by rule id and scope | |
| + |
| +Hard and probe blocks answer `404` with a short plain body and the reference |
| +code in a response header and the body. *My call:* a 404 is what the app would |
| +have said for a path it does not have, so a real visitor who hits one loses |
| +nothing, and a scanner learns nothing. Attack and tenant blocks answer `403` |
| +with a small page naming the platform as the blocker and showing the reference |
| +code. Its wording goes through copy review when built. |
| + |
| +### 5.2 Hard tier |
| + |
| +Blocked for every app, whether or not the file exists, case-insensitively, on |
| +the decoded and dot-segment-resolved path, at any depth: |
| + |
| +- `.env` and `.env.*` path segments. |
| +- `.git`, `.svn`, `.hg` segments; `.git-credentials`. |
| +- `.aws`, `.ssh`, `.docker`, `.kube` segments; `.npmrc`, `.pypirc`, |
| + `.netrc`, `.htpasswd`. |
| +- Private key and credential files: `id_rsa`, `id_ed25519` and similar, and the |
| + credential-file names the evidence shows (`credentials.json`, |
| + `gcp-credentials.json`, `firebase-adminsdk*.json`), as whole segments only. |
| +- Database dumps by extension: `.sql`, `.sql.gz`, `.sqlite`, `.db` dumps at a |
| + path, and backup suffixes on source files (`*.php.save`, `*.bak`, `*.old`) |
| + where the evidence supports them. |
| +- Path traversal: any `..` segment after percent-decoding (including `%2e%2e`, |
| + `%2f` and `%5c` separators). Browsers resolve dot segments before sending, so |
| + a real visitor never sends one. Filenames merely containing `..` (`photo..jpg`) |
| + are not segments and pass. |
| +- `/proc/self/` anywhere in the decoded path, and Vite's `/@fs/` dev door. |
| + |
| +`/.well-known/` is exempt as a whole: it is a legitimate dot-prefixed path (ACME, |
| +`security.txt`, app links). |
| + |
| +*My call:* the exact list is the builder's, drawn from §3 and the appendix, |
| +under one rule: a hard entry must be something no deployed app should ever |
| +serve to the public. Anything a real app might legitimately serve goes in the |
| +probe tier instead. |
| + |
| +**Deploy-time warning.** When a release contains a file matching the hard tier, |
| +the deploy reply names each one and says it will not be served. Where the |
| +platform can see the file tree (static sites; rootfs images once #366 lands), |
| +the check is exact. For other app kinds there is nothing to inspect and no |
| +warning. |
| + |
| +**Hard override.** Serving one of these needs a separate, explicitly named |
| +command (or console action) for one exact path on one app. It cannot take a |
| +prefix or pattern. It is logged with who set it, and the console shows a |
| +standing warning on the app while it exists. |
| + |
| +### 5.3 Probe tier and existence |
| + |
| +The probe catalogue groups paths by the technology they target: WordPress, PHP |
| +(scripts and webshells), phpMyAdmin, PHP frameworks, Java/Spring, CGI and |
| +routers, server-status pages, cloud metadata, admin panels. Each entry matches |
| +**path structure**, never a word inside a path: an exact path, a path prefix on |
| +segment boundaries, or a final-segment extension. `/wp-content/uploads/x.png` |
| +is not `/wp-login.php`, and `/assets/admin/app.js` is not `/admin/login`. |
| + |
| +A probe block only fires when the app would not have served the path. How the |
| +edge knows: |
| + |
| +- **Static sites: exact.** The edge holds the site's file tree |
| + (`internal/edge/static.go`). A path that is in the tree is served. |
| +- **Framework apps: the route list.** The framework publishes the app's routes |
| + in the release manifest at build time and the edge checks against it. A |
| + catch-all route (a single-page app answering every path) counts as unknown, |
| + not as "everything exists". This is a request to the CARLOS framework |
| + (filed separately; §9), not something the platform guesses. |
| +- **Everything else** (other binaries; rootfs apps once #366 lands, which can |
| + genuinely be WordPress): unknown. Three answers: |
| + - A **per-app technology switch** ("this app is PHP", "this app is WordPress") |
| + turns off the matching groups for that app. For rootfs apps, *my call:* |
| + detect the obvious cases at deploy time (a `wp-config.php`, `*.php` files) |
| + and set the switch automatically, telling the developer. |
| + - **Admin panels are never blocked on an unknown app.** `/admin` is a common |
| + real route. That group only feeds the probe-burst limit (§5.5). |
| + - **Learning during calibration.** While a group is in watch mode fleet-wide |
| + (§8), probe-looking requests are forwarded and the response recorded. A |
| + 2xx or 3xx answer exempts that path prefix for that app automatically and |
| + tells the developer. Learned exemptions persist after blocking starts. |
| + |
| +**The catalogue** is a versioned data file in the platform repo, seeded from §3 |
| +and the appendix, plus public scanner wordlists where their licence allows. It |
| +is refreshed from our own request records once those exist, and ships with |
| +releases. *My call:* the file format and where it lives are the builder's. |
| + |
| +Probe and hard checks run **before the activator and before the cache**, as the |
| +password gate does (`internal/edge/edge.go:1688`), so a probe never wakes an |
| +instance and a stale cache entry can never serve a leak. |
| + |
| +### 5.4 Attack tier: the OWASP 80/20 |
| + |
| +*My call:* use **Coraza** (the OWASP Go engine, Apache 2.0) with the **OWASP Core |
| +Rule Set at paranoia level 1**, in anomaly-scoring mode, rather than writing our |
| +own detectors. CRS's SQL and XSS detection parse syntax (libinjection-style), |
| +which is exactly what §3's misfire list demands: `european-union` is not SQL. |
| + |
| +- **A benchmark gate comes first.** The builder measures added latency and |
| + memory at the edge on representative traffic before anything else. If Coraza |
| + cannot meet a budget a visitor would not notice on an ordinary page view, stop |
| + and report back rather than switching to a home-grown matcher. *My call for |
| + the budget:* under 1 ms added at p50 for a GET with no body. |
| +- **Score threshold.** CRS blocks when a request's combined anomaly score crosses |
| + a threshold. *My call:* run calibration at the CRS default and record every |
| + score, so the promoted threshold is chosen from data. |
| +- **Bodies.** Inspect request bodies up to a cap (*my call:* 128 KiB), skip file |
| + contents in multipart uploads, and pass anything beyond the cap uninspected |
| + rather than buffering it. A body cap never refuses a request by itself. |
| +- **Per-rule promotion.** Every CRS rule starts in watch mode. After calibration |
| + (§8), the rules that matched nothing legitimate become block-by-default for |
| + every app. The rest stay watch-only; an app can opt into blocking them. |
| +- **Per-app strictness.** An app can turn the attack tier off, keep the |
| + platform default, or opt into blocking every PL1 rule. *My call:* higher |
| + paranoia levels are not offered in v1. |
| +- **Placement.** After the cache lookup (a cache hit never reaches the app, so |
| + an attack payload in it is harmless), immediately before the activator or |
| + proxy, next to #358's check. |
| +- **Updates.** CRS is pinned to a version in the platform and updated through |
| + ordinary releases. |
| + |
| +### 5.5 Probe-burst limit |
| + |
| +A client that collects many "not found" answers on one app in a short window |
| +(hard and probe blocks, and the app's own 404s) is a scanner working through a |
| +list. Once it crosses the threshold, its further requests **to paths in the |
| +probe catalogue (including admin panels and the unknown-existence groups)** are |
| +refused for a cool-down, without a wake. Its other requests pass. |
| + |
| +This stops bulk probing cold without ever refusing a shared address's real |
| +users their real pages, honouring #358's ruling that a shared address is never |
| +limited pre-emptively. *My call:* 60 not-founds in 60 s, a 10-minute cool-down, |
| +keyed by `clientIP` truncated as #358 does (/32 IPv4, /64 IPv6); the builder |
| +checks both numbers against request records before shipping. |
| + |
| +### 5.6 The "under attack" switch |
| + |
| +A per-app switch, set by CLI or console and live within seconds, for an incident: |
| + |
| +- A browser navigation without a valid pass gets a small interstitial page that |
| + solves a SHA-256 proof-of-work puzzle in the background and continues. The |
| + pass is a signed, short-lived cookie bound to the host. No CAPTCHA. |
| +- Difficulty is set by the server, sealed into the challenge, and rises with |
| + load. |
| +- Cache hits are served without a challenge (they cost the instance nothing). |
| +- Paths the developer marks as non-browser (APIs, webhooks) are not challenged. |
| + #358's limits still apply to them. |
| +- Visitors without JavaScript see a plain explanation. This is an emergency |
| + mode, and that cost is accepted. |
| + |
| +*My call:* other cookieless non-navigation requests (fetches without a pass) get |
| +`429` with `Retry-After`. The builder settles the exact navigation test (for |
| +example `Sec-Fetch-Mode: navigate`, or `Accept: text/html` on a GET) and the pass |
| +lifetime. |
| + |
| +## 6. Tenant rules |
| + |
| +### 6.1 What a rule is |
| + |
| +Rules belong to an **app** and apply to every instance of it, optionally scoped |
| +to environments (`registry.Route.Environment`). A rule has: |
| + |
| +- **Match:** any combination of method, path (exact, prefix, or RE2 regular |
| + expression), query parameter name or value, request header (user agent |
| + included), client address or range, and country (GeoIP, #241). Request |
| + bodies are out of v1. |
| +- **Action:** |
| + - **block**: refuse with the tenant-block page and a reference code. |
| + - **watch**: count and record only. |
| + - **allow**: an exception. It names what it exempts (a probe group, an attack |
| + rule id, a tenant rule by name, or everything except the hard tier) within |
| + its match scope. An allow can never exempt the hard tier. |
| +- **A name**, unique within the app, so records, the console and allow rules |
| + can refer to it. |
| + |
| +*My call:* a small, readable rule syntax of our own, not ModSecurity's SecLang, |
| +which is powerful but hard to read and easy to get wrong. The builder designs |
| +it. Illustration only: |
| + |
| +``` |
| +block bad-export path ^/api/export country != GB |
| +watch odd-agent header User-Agent ~ "sqlmap|nikto" |
| +allow search-sql rule 942100 path /search |
| +``` |
| + |
| +**Order:** the hard tier first (only a hard override suppresses it), then allow |
| +rules, then tenant blocks, probe tier and attack tier. A matching allow |
| +suppresses exactly what it names. |
| + |
| +**Rate limits are not a tenant rule action.** Per-path rate settings live in |
| +#358's per-instance path exceptions. |
| + |
| +### 6.2 File and console: a union |
| + |
| +The platform stores one rule set per app. Every rule records its origin: |
| +**file** or **console**. |
| + |
| +- **The file** lives in the app's repo. One CLI command pushes it; the push |
| + replaces exactly the app's file-origin rules and leaves console rules alone. |
| + So a rule someone added in the console during an incident survives the next |
| + push. |
| +- **The console** lists all of an app's rules. File rules are read-only there |
| + (edit them in the repo). Console rules can be added, edited and deleted in |
| + place. "Copy as file text" on a console rule gives the line to paste into |
| + the file once the emergency is over; when the file containing it is pushed, |
| + the console rule of the same name is retired. |
| +- **Deploys never touch rules.** A deploy can never quietly undo an emergency |
| + rule. |
| +- **History:** every change (who, when, origin, before and after) is recorded |
| + and visible on the app. |
| + |
| +*My call:* when a file push introduces a rule whose name matches a live console |
| +rule, the file rule replaces it and the reply says so. Two different rules with |
| +the same name across origins are never both live. |
| + |
| +### 6.3 Live in seconds |
| + |
| +Rules are validated at the console (syntax, RE2 compile, size limits, scope |
| +belongs to the app) and refused there with a precise error, never half-applied. |
| +Accepted rules converge to every box serving the app within seconds, by the |
| +same route-property path the password gate uses and the change index made fast |
| +(`docs/superpowers/specs/2026-10-21-s3-change-index-design.md`). Deleting a |
| +rule is just as fast. *My call:* limits of 200 rules per app and 4 KiB per |
| +regular expression; the builder may adjust them from benchmarks. |
| + |
| +### 6.4 Seeing what a rule does |
| + |
| +- **A new rule can start in watch mode**, and the CLI reply and console show |
| + its match counts directly. Nobody has to go looking for a report. (A tenant |
| + app that tried a shadow-measurement phase found nobody read the reports.) |
| +- **Backtest:** where request records hold the fields a rule matches on, the |
| + push reply says how many of the last 7 days' requests it would have matched, |
| + with a few examples. *My call:* this is in v1 only if records carry the |
| + fields cheaply. Otherwise it is the first follow-up. |
| + |
| +## 7. Request records |
| + |
| +This comes first: without it nothing can be calibrated, explained or |
| +backtested. |
| + |
| +- **Blocked and watched requests** each get a record: time, reference code, |
| + host, app, environment, tier, rule id or name, action, method, path, query |
| + parameter **names**, user-agent family, country, a truncated client address |
| + (/24 IPv4, /48 IPv6) plus a keyed hash of the full key for correlation, and |
| + the matched fragment. |
| +- **Redaction:** the matched fragment is truncated (*my call:* 64 bytes) and |
| + omitted entirely when it comes from a cookie, an `Authorization` header, or a |
| + parameter whose name looks secret (`password`, `token`, `secret`, `key`, |
| + `session` and similar). |
| +- **Everything else** is counted in memory per app, per minute (status class, |
| + method, probe-catalogue hits, wake caused or not) and flushed in batches. No |
| + storage write per request, ever, and a flood must not turn records into its |
| + own load: under pressure records are sampled, and the counts say how many |
| + were dropped. |
| +- **Shipping** uses the existing per-app log pipeline (`internal/applog`, |
| + `StreamEdge`) and its budgets, so they are readable with `carlos logs` and in |
| + the console. *My call:* 14 days' retention. |
| +- **Reference codes** are short, unguessable, and resolve (for the app's members |
| + only) to the record: which rule fired, on what, and the one-line exception |
| + that would allow it. |
| +- **Calibration needs more:** during the watch window (§8), probe-catalogue and |
| + attack-tier matches also record the upstream response status, so learned |
| + exemptions (§5.3) and promotion decisions have the evidence. |
| + |
| +Fixing the replica-sync log volume that limits journald retention to ~12 hours |
| +on the busiest box is a separate, small piece of work, worth doing alongside. |
| + |
| +## 8. Rollout and calibration |
| + |
| +1. **Records first**, on dev boxes, then the flagship. |
| +2. **Hard tier and probe tier in watch mode** fleet-wide on the flagship (and |
| + any self-hosted deployment that takes the release) for a calibration window. |
| + *My call:* at least 14 days, ended by a written report. |
| +3. **Promotion by review.** We read every distinct (rule, path) match class in |
| + the report, assisted by flags for likely-real traffic (a same-site Referer, |
| + browser fetch headers, the same client also getting real pages). A rule or |
| + catalogue entry goes to block-by-default only if nothing in its class looks |
| + legitimate. Paul approves the promotion list. |
| +4. **Tenant rules** ship once records and the convergence path are in place. |
| +5. **Attack tier** follows the same watch, report, review and promotion cycle, |
| + after its benchmark gate. |
| +6. **Under attack** last. |
| + |
| +Each step is its own implementation plan, rolled dev first, then the flagship. |
| + |
| +## 9. Requests to other pieces |
| + |
| +- **CARLOS framework:** publish the app's route list in the release manifest at |
| + build time, with catch-all routes marked (§5.3). Filed as a request on the |
| + framework, not worked around. |
| +- **Rootfs apps (#366):** the deploy step needs to expose the image's file list |
| + (or the platform's scan of it) for the hard-tier warning and the PHP and |
| + WordPress switch. |
| +- **#358:** shares the client key, the place in the edge and possibly the |
| + per-host counters. The two must not double-count or disagree on the key. |
| + |
| +## 10. Left to the builder |
| + |
| +- The exact hard-tier list and the probe catalogue's format, location and |
| + matching implementation, within §5.2 and §5.3's rules. |
| +- The rule syntax (§6.1), the CLI verbs and console screens. All user-facing |
| + words (verbs, flags, block pages, deploy warnings, console text) go through |
| + copy review. |
| +- Where exactly each check sits in `ServeHTTP`, within the constraints stated: |
| + hard and probe checks before the activator and the cache; attack checks after |
| + the cache and before the activator or proxy; every check evaluated exactly |
| + once per request, including requests forwarded between boxes, on a box that |
| + holds the app's current rules. |
| +- The calibration report's format. |
| +- Every number marked *my call*, checked against benchmarks and records. |
| + |
| +## Appendix: seed catalogue (from §3's 7 days) |
| + |
| +**Hard tier candidates:** `/.env`, `/.env.local`, `/.env.production`, |
| +`/.env.backup`, `/.env.old`, `/.env.development`, `/.env.bak`, `/.env.prod`, |
| +`/.env.save`, `/<dir>/.env` (backend, app, api, admin, config, core, src, |
| +laravel, server, public, …), `/.git/config`, `/.git/HEAD`, `/.git-credentials`, |
| +`/.aws/credentials`, `/.aws/config`, `/.docker/config.json`, `/credentials.json`, |
| +`/gcp-credentials.json`, `/firebase-adminsdk.json`, `/@fs/.env`, |
| +`/@fs/proc/self/environ`, `/static/..%2f.env`, `/db.sql`, `*.php.save`. |
| + |
| +**Probe tier candidates by group:** |
| + |
| +- **WordPress:** `/wp-login.php`, `/xmlrpc.php`, `/wp-links-opml.php`, |
| + `/wp-admin/`, `/wp-json/` (also under `/site`, `/news`, `/cms`, `/blog`, `/wp`, |
| + `/wordpress`, `/press`), `/wordpress/`, |
| + `/wp-content/plugins/*/wp_filemanager.php`. Not `/wp-content/uploads/` images. |
| +- **PHP:** any `*.php` final segment on an app not marked PHP; `/_profiler/phpinfo`. |
| +- **PHP frameworks:** `/vendor/phpunit/`, `/_ignition/`, `/telescope`, |
| + `/_debugbar`, `/vendor/composer/installed.json`, ThinkPHP `?s=/index/\think…`. |
| +- **phpMyAdmin:** `/phpmyadmin/`, `/pma/` (as whole leading segments only). |
| +- **Java/Spring:** `/actuator`, `/actuator/*`, `*.action`, `*.do`, `*.jsp`. |
| +- **CGI, routers, IoT:** `/cgi-bin/`, `/setup.cgi`, `/HNAP1`, `/GponForm/`. |
| +- **Server status and metadata:** `/server-status`, `/server-info`, |
| + `/latest/meta-data/`. |
| +- **Dev and config files:** `/docker-compose.yml`, `/Dockerfile`, |
| + `/.gitlab-ci.yml`. |
| +- **Admin panels (burst limit only, §5.5):** `/admin`, `/admin/login`, |
| + `/admin/settings`, `/admin/dashboard`, `/administrator/`. |