carlos / platform Public

Merge branch 'carlos-static-redirects' into main

carlos-static-redirects: static sites honour a _redirects file (redirects, rewrites, a not-found page). Census overlay static-redirects proposed on #280.
james@keymail.dev 0d9e81dc31dc3c9fab20736e8c7d168fed8b7c8c parent d18e619parent ff350c2
6 files changed, +928 −33
  • docs/site/cli.md +38 −0
  • internal/edge/redirects.go +418 −0
  • internal/edge/redirects_test.go +319 −0
  • internal/edge/static.go +127 −33
  • internal/provision/plan14_root_reads_test.go +15 −0
  • internal/provision/testdata/plan14-root-reads-static-redirects.tsv +11 −0

CI passed — run details

CI log
=== gate ===
.amadan/test-gate.sh
=== launchctx (1 s)
ok  	amadan.net/carlos/platform.git/launchctx	0.012s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
=== vet (19 s)
=== gofmt (0 s)
gofmt -l .: clean
=== rest (177 s)
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.668s
ok  	github.com/carlosframework/platform/cmd/moveprobe	11.899s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.786s
ok  	github.com/carlosframework/platform/internal/activator	61.586s
ok  	github.com/carlosframework/platform/internal/agentsock	0.033s
ok  	github.com/carlosframework/platform/internal/alerting	0.009s
ok  	github.com/carlosframework/platform/internal/analytics	0.033s
ok  	github.com/carlosframework/platform/internal/appconfig	3.063s
ok  	github.com/carlosframework/platform/internal/apperr	0.109s
ok  	github.com/carlosframework/platform/internal/applog	0.267s
ok  	github.com/carlosframework/platform/internal/awsinv	0.010s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.009s
ok  	github.com/carlosframework/platform/internal/boxid	1.047s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.019s
ok  	github.com/carlosframework/platform/internal/caps	0.019s
ok  	github.com/carlosframework/platform/internal/certwarm	0.005s
ok  	github.com/carlosframework/platform/internal/cgroup	0.825s
ok  	github.com/carlosframework/platform/internal/changelog	0.464s
ok  	github.com/carlosframework/platform/internal/claimactivation	1.312s
ok  	github.com/carlosframework/platform/internal/clamd	0.068s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	8.136s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.012s
ok  	github.com/carlosframework/platform/internal/dbquery	0.010s
ok  	github.com/carlosframework/platform/internal/dbreads	2.057s
?   	github.com/carlosframework/platform/internal/dbreads/dbreadstest	[no test files]
ok  	github.com/carlosframework/platform/internal/dbverify	0.009s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.009s
ok  	github.com/carlosframework/platform/internal/docsite	0.008s
ok  	github.com/carlosframework/platform/internal/domains	2.900s
ok  	github.com/carlosframework/platform/internal/doorkey	0.012s
ok  	github.com/carlosframework/platform/internal/economics	0.004s
ok  	github.com/carlosframework/platform/internal/edge	46.207s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.008s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.138s
ok  	github.com/carlosframework/platform/internal/gate	1.657s
ok  	github.com/carlosframework/platform/internal/genselect	0.024s
ok  	github.com/carlosframework/platform/internal/geo	0.011s
ok  	github.com/carlosframework/platform/internal/geoip	0.030s
ok  	github.com/carlosframework/platform/internal/historyreadback	0.162s
ok  	github.com/carlosframework/platform/internal/host	147.274s
ok  	github.com/carlosframework/platform/internal/hostactors	5.987s
ok  	github.com/carlosframework/platform/internal/hostactors/awsmembership	0.012s
ok  	github.com/carlosframework/platform/internal/hostbox	3.133s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.799s
ok  	github.com/carlosframework/platform/internal/hostenv	0.015s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.015s
ok  	github.com/carlosframework/platform/internal/hostexec	5.438s
ok  	github.com/carlosframework/platform/internal/hostfence	0.052s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.334s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.023s
ok  	github.com/carlosframework/platform/internal/hostguard	1.037s
ok  	github.com/carlosframework/platform/internal/hostops	2.048s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.012s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.015s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.666s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.204s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.015s
ok  	github.com/carlosframework/platform/internal/infraview	0.015s
ok  	github.com/carlosframework/platform/internal/instancectl	5.236s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.837s
ok  	github.com/carlosframework/platform/internal/instancefs	0.023s
ok  	github.com/carlosframework/platform/internal/instancekey	0.019s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.025s
ok  	github.com/carlosframework/platform/internal/layout	1.818s
ok  	github.com/carlosframework/platform/internal/lbhealth	0.017s
ok  	github.com/carlosframework/platform/internal/lease	10.221s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.590s
ok  	github.com/carlosframework/platform/internal/ledger	0.027s
ok  	github.com/carlosframework/platform/internal/mail	0.280s
ok  	github.com/carlosframework/platform/internal/maintenance	0.124s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.576s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.112s
ok  	github.com/carlosframework/platform/internal/metrics	0.010s
ok  	github.com/carlosframework/platform/internal/moves	22.527s
ok  	github.com/carlosframework/platform/internal/moves/archive	1.040s
ok  	github.com/carlosframework/platform/internal/objstore	0.321s
ok  	github.com/carlosframework/platform/internal/ops	0.236s
ok  	github.com/carlosframework/platform/internal/park	0.258s
ok  	github.com/carlosframework/platform/internal/plans	0.023s
ok  	github.com/carlosframework/platform/internal/privatefile	0.004s
ok  	github.com/carlosframework/platform/internal/provision	24.230s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.008s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	5.678s
ok  	github.com/carlosframework/platform/internal/relay	7.241s
ok  	github.com/carlosframework/platform/internal/release	26.916s
ok  	github.com/carlosframework/platform/internal/rollout	3.833s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.004s
ok  	github.com/carlosframework/platform/internal/rootaws	0.013s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.069s
ok  	github.com/carlosframework/platform/internal/rootstate	0.027s
ok  	github.com/carlosframework/platform/internal/runs	0.567s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.664s
ok  	github.com/carlosframework/platform/internal/seeds	0.019s
ok  	github.com/carlosframework/platform/internal/sim/engines	2.634s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	12.006s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.644s
ok  	github.com/carlosframework/platform/internal/status	0.066s
ok  	github.com/carlosframework/platform/internal/steering	0.007s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.313s
ok  	github.com/carlosframework/platform/internal/svcrole	0.003s
ok  	github.com/carlosframework/platform/internal/telemetry	0.005s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.008s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.008s
ok  	github.com/carlosframework/platform/internal/usage	0.010s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.688s
=== internal_console@0 (87 s)
ok  	github.com/carlosframework/platform/internal/console	85.228s
=== internal_console@1 (42 s)
ok  	github.com/carlosframework/platform/internal/console	40.642s
=== internal_console@2 (45 s)
ok  	github.com/carlosframework/platform/internal/console	43.172s
=== internal_console@3 (43 s)
ok  	github.com/carlosframework/platform/internal/console	41.910s
=== internal_console@4 (46 s)
ok  	github.com/carlosframework/platform/internal/console	44.451s
=== internal_console@5 (42 s)
ok  	github.com/carlosframework/platform/internal/console	40.264s
=== internal_console@6 (40 s)
ok  	github.com/carlosframework/platform/internal/console	38.371s
=== internal_console@7 (37 s)
ok  	github.com/carlosframework/platform/internal/console	35.362s
=== root@0 (112 s)
ok  	github.com/carlosframework/platform	110.907s
=== root@1 (90 s)
ok  	github.com/carlosframework/platform	88.227s
=== root@2 (81 s)
ok  	github.com/carlosframework/platform	79.736s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 50528418 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: downloading https://github.com/minio/minio/releases/download/RELEASE.2025-09-07T16-13-09Z/minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z
  minio: downloaded, digest verified
  mc: downloading https://github.com/minio/mc/releases/download/RELEASE.2025-08-13T08-35-41Z/mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z
  mc: downloaded, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.025s
=== RUN   TestT4StorageProbeMinio
2026/10/23 19:36:19 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18E13FB68BB58BB0, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.025s
=== RUN   TestS3StoreMinioContract
=== RUN   TestS3StoreMinioContract/conditional_get_and_etag_listing
=== RUN   TestS3StoreMinioContract/bump_through_every_mutation_path
--- PASS: TestS3StoreMinioContract (0.03s)
    --- PASS: TestS3StoreMinioContract/conditional_get_and_etag_listing (0.01s)
    --- PASS: TestS3StoreMinioContract/bump_through_every_mutation_path (0.02s)
PASS
ok  	github.com/carlosframework/platform/internal/release	0.032s
Bucket created successfully `local/dbf-deploy`.
Bucket created successfully `local/dbf-data-ie`.
Bucket created successfully `local/dbf-data-uk`.
local/dbf-deploy versioning is enabled
Created policy `dbf-console` successfully.
Created policy `dbf-ukbox` successfully.
Added user `dbfconsole` successfully.
Added user `dbfukbox` successfully.
Attached Policies: [dbf-console]
To User: dbfconsole
Attached Policies: [dbf-ukbox]
To User: dbfukbox
=== RUN   TestDBForwardMinio
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/23 19:36:22 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=720
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/23 19:36:22 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=370
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/23 19:36:22 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=454
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/23 19:36:22 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=720
2026/10/23 19:36:22 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/23 19:36:22 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/23 19:36:22 console: could not notify member@example.com of a CLI token issue: keymail: lookup member@example.com: keymail: GET /api/federation/lookup?addr=member%40example.com&from=console%40127.0.0.1%3A35103: could not determine target mailbox
    agent.go:123: dbreads: dbr_6izsjauyhgwqjborarqiymfuji: planned 3 files (1544 bytes) and restored to 3 in 10ms
    agent.go:123: dbreads: dbr_6izsjauyhgwqjborarqiymfuji: output 5 bytes in 1 confirmed chunks in 1ms
2026/10/23 19:36:23 console: db read dbr_6izsjauyhgwqjborarqiymfuji: relayed 1 chunks (102 bytes) in 1.004s, ended done
    agent.go:123: dbreads: dbr_izkuh4uapo5fhb2j5xfwsvkroq: planned 1 files (720 bytes) and restored to 1 in 8ms
    agent.go:123: dbreads: dbr_izkuh4uapo5fhb2j5xfwsvkroq: output 5 bytes in 1 confirmed chunks in 1ms
2026/10/23 19:36:24 console: db read dbr_izkuh4uapo5fhb2j5xfwsvkroq: relayed 1 chunks (102 bytes) in 1.004s, ended done
=== RUN   TestDBForwardMinio/by_disconnect
=== NAME  TestDBForwardMinio
    agent.go:123: dbreads: dbr_qsylng4l4tbzckuutdbxrqt3de: planned 3 files (1544 bytes) and restored to 3 in 10ms
2026/10/23 19:36:24 console: db read dbr_qsylng4l4tbzckuutdbxrqt3de: relayed 0 chunks (0 bytes) in 29ms, ended client gone
    agent.go:123: dbreads: dbr_qsylng4l4tbzckuutdbxrqt3de: output 5 bytes in 1 confirmed chunks in 104ms
=== RUN   TestDBForwardMinio/by_timeout
=== NAME  TestDBForwardMinio
    agent.go:123: dbreads: dbr_vhwrll3ttsvnlofmyg2wdkhshe: planned 3 files (1544 bytes) and restored to 3 in 10ms
2026/10/23 19:36:44 console: db read dbr_vhwrll3ttsvnlofmyg2wdkhshe: relayed 0 chunks (0 bytes) in 19.247s, ended no-result
    agent.go:123: dbreads: dbr_vhwrll3ttsvnlofmyg2wdkhshe: output 5 bytes in 1 confirmed chunks in 19.223s
--- PASS: TestDBForwardMinio (21.68s)
    --- PASS: TestDBForwardMinio/by_disconnect (0.24s)
    --- PASS: TestDBForwardMinio/by_timeout (19.36s)
PASS
ok  	github.com/carlosframework/platform/internal/console	21.700s
minio contract: PASS
diff --git a/docs/site/cli.md b/docs/site/cli.md
index f001dd39e..a7b3f8e27 100644
--- a/docs/site/cli.md
+++ b/docs/site/cli.md
@@ -330,6 +330,44 @@ app is served only at its own platform address, so `--host` and
carlos deploy --temporary --ttl 3d ./dist
```
+#### Redirects for a static site
+
+Put a file named `_redirects` at the top of the site directory. It uses the
+format Netlify and Cloudflare Pages use: one rule per line, with a source, a
+destination and an optional status.
+
+```text
+# source destination status
+/old-page /new-page 301
+/blog/* /posts/:splat 301
+/docs/:slug /guide/:slug 302
+/app/* /app/index.html 200
+https://www.example.com/* https://example.com/:splat 301!
+/* /404.html 404
+```
+
+- A source is a path, or a URL with a host. A URL source applies only to
+ requests for that host, so the last-but-one rule sends `www` to the apex.
+- `:name` matches one path segment. A `*` at the end matches the rest of the
+ path, and the destination gets it as `:splat`.
+- The status is 301 when you leave it out. 302, 303, 307 and 308 are also
+ redirects. 200 serves the destination's file at the requested URL, which
+ is how a single-page app serves its client-side routes. 404 serves the
+ destination's file as the not-found page.
+- A 200 or 404 rule takes a path in the same site. A redirect can also go to
+ another host.
+- A request's query string is kept unless the destination has one of its
+ own.
+- When a file exists at the requested path, the file wins. Add `!` to the
+ status (`301!`) to make the rule win.
+- Rules apply in order, and the first rule that matches is used. A line
+ starting with `#` is a comment.
+
+`carlos ship` and `carlos deploy` refuse a `_redirects` file with a line they
+cannot use, and name the line. Netlify's conditions, such as `Country=`, and
+proxying to another host are not supported. A file can have up to 2,000
+rules. The edge does not serve the `_redirects` file itself.
+
#### Two builds at once
When a new build reaches an awake instance, it starts beside the old one,
diff --git a/internal/edge/redirects.go b/internal/edge/redirects.go
new file mode 100644
index 000000000..f38a1d416
--- /dev/null
+++ b/internal/edge/redirects.go
@@ -0,0 +1,418 @@
+// Redirect and rewrite rules for static sites: a `_redirects` file at the root
+// of a shipped tree, in the format Netlify introduced and Cloudflare Pages
+// adopted. One rule per line, "source destination [status]":
+//
+// /old /new 301
+// /blog/* /posts/:splat
+// /docs/:slug /guide/:slug 302
+// /app/* /app/index.html 200
+// https://www.example.com/* https://example.com/:splat 301!
+//
+// The subset served here:
+//
+// - A source is a path, or an http(s) URL whose host must match the
+// request's (the scheme is not compared: the edge serves https only).
+// A segment ":name" captures one path segment; a final "*" captures the
+// rest of the path, "" included, so "/blog/*" also matches "/blog"; a
+// non-empty splat keeps the request's trailing slash.
+// Matching is case-sensitive and ignores a trailing slash.
+// - A destination is a path, or for a redirect an http(s) URL. ":name" and
+// ":splat" are replaced with what the source captured; placeholders in a
+// URL's scheme or host are not. A request's query string is carried over
+// when the destination names none of its own.
+// - The status is 301 (the default), 302, 303, 307 or 308 for a redirect;
+// 200 serves the destination's file at the request's URL (a rewrite) and
+// 404 serves it with a 404 (a custom not-found page). A rewrite's
+// destination is always a file in the same tree, never another host.
+// - A file at the request path wins over every rule (Netlify's
+// "shadowing"), unless the status carries a trailing "!".
+// - The first matching rule applies. A line whose first field starts with
+// "#" is a comment, and so is everything from a field starting with "#".
+//
+// Netlify's conditions (Country=, Language=, Role=...), proxying to another
+// host and the netlify.toml form are not supported. `carlos ship` refuses a
+// tree whose _redirects has a line it cannot use (PackDir), so a site never
+// ships rules the edge would ignore; the edge itself skips such a line and
+// logs it, which only a tree packed by something else can reach.
+package edge
+
+import (
+ "errors"
+ "fmt"
+ "net/url"
+ "path"
+ "strconv"
+ "strings"
+)
+
+// redirectsFile is the rules file's name at the root of a static tree. The
+// file is configuration, not content: the edge never serves it.
+const redirectsFile = "_redirects"
+
+// Limits on one _redirects file. Matching is a scan of every rule on each
+// request the tree cannot answer, so the count is bounded the way Cloudflare
+// Pages bounds it.
+const (
+ redirectsMaxBytes = 256 << 10
+ redirectsMaxRules = 2000
+ redirectsMaxLine = 2048
+)
+
+// redirectRule is one parsed line of a _redirects file.
+type redirectRule struct {
+ line int // 1-based line number, for logs
+ host string // lowercased; "" matches any host
+ segs []string // source path segments; ":name" captures one
+ splat bool // the source ends in "*"
+ prefix string // a URL destination's "scheme://host", never substituted
+ to string // the destination template after prefix
+ status int
+ force bool // "!": applies even when a file exists at the request path
+}
+
+// redirectRules is a parsed _redirects file, immutable once built.
+type redirectRules struct {
+ rules []redirectRule
+}
+
+// isRedirect reports whether status is one a redirect rule may answer with.
+func isRedirect(status int) bool {
+ switch status {
+ case 301, 302, 303, 307, 308:
+ return true
+ }
+ return false
+}
+
+// parseRedirects parses a _redirects file. It returns the rules it could use
+// and an error naming every line it could not; callers decide which matters.
+// Ship refuses on any error, and the edge serves the usable rules.
+func parseRedirects(data []byte) (*redirectRules, error) {
+ if len(data) > redirectsMaxBytes {
+ return nil, fmt.Errorf("%s is %d bytes; the limit is %d", redirectsFile, len(data), redirectsMaxBytes)
+ }
+ rs := &redirectRules{}
+ var errs []error
+ // A byte-order mark is what some editors save at the front; it is not
+ // part of the first rule.
+ text := strings.TrimPrefix(string(data), "\ufeff")
+ for i, line := range strings.Split(text, "\n") {
+ n := i + 1
+ line = strings.TrimSpace(strings.TrimSuffix(line, "\r"))
+ if line == "" || strings.HasPrefix(line, "#") {
+ continue
+ }
+ if len(line) > redirectsMaxLine {
+ errs = append(errs, fmt.Errorf("line %d: longer than %d characters", n, redirectsMaxLine))
+ continue
+ }
+ rule, err := parseRedirectLine(line)
+ if err != nil {
+ errs = append(errs, fmt.Errorf("line %d: %v", n, err))
+ continue
+ }
+ if len(rs.rules) == redirectsMaxRules {
+ errs = append(errs, fmt.Errorf("line %d: more than %d rules", n, redirectsMaxRules))
+ continue
+ }
+ rule.line = n
+ rs.rules = append(rs.rules, rule)
+ }
+ return rs, errors.Join(errs...)
+}
+
+func parseRedirectLine(line string) (redirectRule, error) {
+ var fields []string
+ for _, f := range strings.Fields(line) {
+ if strings.HasPrefix(f, "#") {
+ break // the rest of the line is a comment
+ }
+ fields = append(fields, f)
+ }
+ var rule redirectRule
+ switch {
+ case len(fields) < 2:
+ return rule, errors.New("want a source and a destination")
+ case len(fields) > 3:
+ if strings.Contains(fields[3], "=") {
+ return rule, fmt.Errorf("conditions such as %q are not supported", fields[3])
+ }
+ return rule, fmt.Errorf("unexpected %q after the status", fields[3])
+ }
+
+ rule.status = 301
+ if len(fields) == 3 {
+ st := fields[2]
+ if s, ok := strings.CutSuffix(st, "!"); ok {
+ rule.force, st = true, s
+ }
+ code, err := strconv.Atoi(st)
+ if err != nil || !(isRedirect(code) || code == 200 || code == 404) {
+ return rule, fmt.Errorf("status %q: want 301, 302, 303, 307, 308, 200 or 404, optionally followed by !", fields[2])
+ }
+ rule.status = code
+ }
+
+ src := fields[0]
+ if strings.HasPrefix(src, "http://") || strings.HasPrefix(src, "https://") {
+ u, err := url.Parse(src)
+ if err != nil || u.Host == "" || u.User != nil || u.Port() != "" || u.RawQuery != "" || u.Fragment != "" {
+ return rule, fmt.Errorf("source %q: want a URL with only a host and a path", src)
+ }
+ rule.host = serverName(u.Hostname()) // as requestHost normalizes the request's
+ src = u.Path
+ if src == "" {
+ src = "/"
+ }
+ }
+ if !strings.HasPrefix(src, "/") {
+ return rule, fmt.Errorf("source %q: want a path starting with / or an http(s) URL", fields[0])
+ }
+ if strings.ContainsAny(src, "?#") {
+ return rule, fmt.Errorf("source %q: query strings and fragments are not matched", fields[0])
+ }
+ names := map[string]bool{}
+ trimmed := strings.Trim(src, "/")
+ for _, seg := range strings.Split(trimmed, "/") {
+ switch {
+ case seg == "":
+ if trimmed != "" {
+ return rule, fmt.Errorf("source %q: empty path segment", fields[0])
+ }
+ case rule.splat:
+ return rule, fmt.Errorf("source %q: * must be the last segment", fields[0])
+ case seg == "*":
+ rule.splat = true
+ case strings.Contains(seg, "*"):
+ return rule, fmt.Errorf("source %q: * must be a whole segment", fields[0])
+ case strings.HasPrefix(seg, ":"):
+ name := seg[1:]
+ if !validPlaceholder(name) || name == "splat" {
+ return rule, fmt.Errorf("source %q: bad placeholder %q", fields[0], seg)
+ }
+ if names[name] {
+ return rule, fmt.Errorf("source %q: placeholder %q appears twice", fields[0], seg)
+ }
+ names[name] = true
+ rule.segs = append(rule.segs, seg)
+ default:
+ // Requests match decoded, as files are found, so a literal
+ // segment is decoded here to meet them: "/caf%C3%A9" matches a
+ // request for /café. A decoded "/" would be two segments, and
+ // no request path has one inside a segment.
+ dec, err := url.PathUnescape(seg)
+ if err != nil || strings.Contains(dec, "/") {
+ return rule, fmt.Errorf("source %q: bad percent-encoding in %q", fields[0], seg)
+ }
+ if strings.HasPrefix(dec, ":") {
+ return rule, fmt.Errorf("source %q: %q would read as a placeholder", fields[0], seg)
+ }
+ if dec == "." || dec == ".." {
+ return rule, fmt.Errorf("source %q: dot segments never match", fields[0])
+ }
+ rule.segs = append(rule.segs, dec)
+ }
+ }
+
+ dst := fields[1]
+ switch {
+ case strings.HasPrefix(dst, "http://") || strings.HasPrefix(dst, "https://"):
+ if !isRedirect(rule.status) {
+ return rule, fmt.Errorf("destination %q: a %d serves a file from this site, so it takes a path, not a URL", dst, rule.status)
+ }
+ u, err := url.Parse(dst)
+ if err != nil || u.Host == "" || u.User != nil {
+ return rule, fmt.Errorf("destination %q: not a usable URL", dst)
+ }
+ i := len(u.Scheme) + len("://") + len(u.Host)
+ if !strings.EqualFold(dst[:i], u.Scheme+"://"+u.Host) {
+ return rule, fmt.Errorf("destination %q: not a usable URL", dst)
+ }
+ rule.prefix, rule.to = dst[:i], dst[i:]
+ if rule.to != "" && !strings.ContainsAny(rule.to[:1], "/?#") {
+ return rule, fmt.Errorf("destination %q: not a usable URL", dst)
+ }
+ case strings.HasPrefix(dst, "//") || strings.HasPrefix(dst, "/\\"):
+ return rule, fmt.Errorf("destination %q: name another host with https://", dst)
+ case strings.HasPrefix(dst, "/"):
+ rule.to = dst
+ default:
+ return rule, fmt.Errorf("destination %q: want a path starting with / or an http(s) URL", dst)
+ }
+ return rule, nil
+}
+
+// validPlaceholder: a placeholder name is letters, digits and underscores.
+func validPlaceholder(name string) bool {
+ if name == "" {
+ return false
+ }
+ for _, c := range name {
+ if !(c == '_' || c >= '0' && c <= '9' || c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z') {
+ return false
+ }
+ }
+ return true
+}
+
+// match reports whether the rule applies to a request for host whose cleaned
+// path splits into reqSegs, and returns what the source captured.
+func (rule *redirectRule) match(host string, reqSegs []string) (map[string]string, bool) {
+ if rule.host != "" && rule.host != host {
+ return nil, false
+ }
+ if len(reqSegs) < len(rule.segs) || !rule.splat && len(reqSegs) != len(rule.segs) {
+ return nil, false
+ }
+ var caps map[string]string
+ for i, seg := range rule.segs {
+ if strings.HasPrefix(seg, ":") {
+ if caps == nil {
+ caps = map[string]string{}
+ }
+ caps[seg[1:]] = reqSegs[i]
+ } else if seg != reqSegs[i] {
+ return nil, false
+ }
+ }
+ if rule.splat {
+ if caps == nil {
+ caps = map[string]string{}
+ }
+ caps["splat"] = strings.Join(reqSegs[len(rule.segs):], "/")
+ }
+ return caps, true
+}
+
+// expand substitutes the captures into the destination template. With escape
+// set, values are percent-encoded for a Location header: per segment in the
+// path, as a query value after "?". Without it they go in raw, for a rewrite's
+// file lookup. A ":name" the source did not capture stays as written.
+func expand(t string, caps map[string]string, escape bool) string {
+ var b strings.Builder
+ inQuery := false
+ for i := 0; i < len(t); {
+ c := t[i]
+ if c == '?' || c == '#' {
+ inQuery = true
+ }
+ if c != ':' {
+ b.WriteByte(c)
+ i++
+ continue
+ }
+ j := i + 1
+ for j < len(t) && validPlaceholder(t[j:j+1]) {
+ j++
+ }
+ v, ok := caps[t[i+1:j]]
+ if !ok {
+ b.WriteString(t[i:j])
+ i = j
+ continue
+ }
+ switch {
+ case !escape:
+ b.WriteString(v)
+ case inQuery:
+ b.WriteString(url.QueryEscape(v))
+ default:
+ parts := strings.Split(v, "/")
+ for k, p := range parts {
+ parts[k] = url.PathEscape(p)
+ }
+ b.WriteString(strings.Join(parts, "/"))
+ }
+ i = j
+ }
+ return b.String()
+}
+
+// redirectAction is what a matched rule asks the handler to do.
+type redirectAction struct {
+ status int
+ location string // a redirect's Location
+ file string // a rewrite's cleaned, rooted path within the tree
+}
+
+// resolve finds the first rule that applies to r's host and the cleaned path
+// p. exists is whether the tree has a file at p, which shadows every rule not
+// forced. rawPath and rawQuery are the request's own, used to refuse a
+// redirect to the URL being requested, which would loop.
+func (rs *redirectRules) resolve(host, p, rawPath, rawQuery string, exists bool) (redirectAction, bool) {
+ if rs == nil || len(rs.rules) == 0 {
+ return redirectAction{}, false
+ }
+ var reqSegs []string
+ if p != "/" {
+ reqSegs = strings.Split(p[1:], "/")
+ }
+ for i := range rs.rules {
+ rule := &rs.rules[i]
+ if exists && !rule.force {
+ continue
+ }
+ caps, ok := rule.match(host, reqSegs)
+ if !ok {
+ continue
+ }
+ if sp := caps["splat"]; sp != "" && strings.HasSuffix(rawPath, "/") {
+ // The cleaned path lost it, and a page moved from /about/ to
+ // /about resolves its relative links against the wrong directory.
+ caps["splat"] = sp + "/"
+ }
+ if !isRedirect(rule.status) {
+ // The template's own query and fragment are cut BEFORE the
+ // captures go in: a decoded capture can hold "?" or "#", and it
+ // is part of the file's name, not the end of it.
+ to := rule.to
+ if k := strings.IndexAny(to, "?#"); k >= 0 {
+ to = to[:k]
+ }
+ return redirectAction{status: rule.status, file: path.Clean("/" + expand(to, caps, false))}, true
+ }
+ // loc is the destination after any scheme://host: path, query.
+ loc := expand(rule.to, caps, true)
+ frag := ""
+ if k := strings.IndexByte(loc, '#'); k >= 0 {
+ loc, frag = loc[:k], loc[k:]
+ }
+ if rule.prefix == "" {
+ // Cleaned here, as http.Redirect would clean it, so the loop
+ // check below sees the Location that is actually sent. Clean
+ // also folds the "//" an empty capture can leave at the front,
+ // which a browser would read as another host.
+ pth, q, hasQ := strings.Cut(loc, "?")
+ trailing := strings.HasSuffix(pth, "/")
+ pth = path.Clean("/" + pth)
+ if trailing && pth != "/" {
+ pth += "/"
+ }
+ loc = pth
+ if hasQ {
+ loc += "?" + q
+ }
+ }
+ if !strings.Contains(loc, "?") && rawQuery != "" {
+ loc += "?" + rawQuery
+ }
+ if rule.prefix == "" || hostOf(rule.prefix) == host {
+ pth, q, _ := strings.Cut(loc, "?")
+ if pth == "" {
+ pth = "/"
+ }
+ if u, err := url.Parse(pth); err == nil && u.Path == rawPath && q == rawQuery {
+ continue // a redirect to this very URL
+ }
+ }
+ return redirectAction{status: rule.status, location: rule.prefix + loc + frag}, true
+ }
+ return redirectAction{}, false
+}
+
+// hostOf is the host of a "scheme://host[:port]" prefix, normalized as
+// requestHost normalizes a request's.
+func hostOf(prefix string) string {
+ _, h, _ := strings.Cut(prefix, "://")
+ return requestHost(h)
+}
diff --git a/internal/edge/redirects_test.go b/internal/edge/redirects_test.go
new file mode 100644
index 000000000..8495a3537
--- /dev/null
+++ b/internal/edge/redirects_test.go
@@ -0,0 +1,319 @@
+package edge
+
+import (
+ "context"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/registry"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+func TestParseRedirectsAcceptsTheSupportedForms(t *testing.T) {
+ rs, err := parseRedirects([]byte(`# a comment
+
+/old /new
+/blog/* /posts/:splat 302
+/docs/:slug /guide/:slug 307 # trailing comment
+/app/* /app/index.html 200
+/* /404.html 404
+/forced /elsewhere 301!
+https://www.example.com/* https://example.com/:splat 301!
+/ext https://example.org/x?y=1 308
+` + "/crlf /there 303\r\n"))
+ if err != nil {
+ t.Fatalf("parseRedirects: %v", err)
+ }
+ if _, err := parseRedirects([]byte("\ufeff/bom /ok\n")); err != nil {
+ t.Fatalf("a file starting with a byte-order mark: %v", err)
+ }
+ if got := len(rs.rules); got != 9 {
+ t.Fatalf("parsed %d rules, want 9", got)
+ }
+ if r := rs.rules[0]; r.status != 301 || r.force || r.line != 3 {
+ t.Fatalf("rule 0 = %+v, want a 301 from line 3, not forced", r)
+ }
+ if r := rs.rules[6]; r.host != "www.example.com" || !r.force || !r.splat || r.prefix != "https://example.com" {
+ t.Fatalf("host rule = %+v", r)
+ }
+}
+
+func TestParseRedirectsRefusesWhatItCannotServe(t *testing.T) {
+ for _, line := range []string{
+ "/only-a-source",
+ "/a /b 301 Country=us",
+ "/a /b 301 extra",
+ "/a /b 418",
+ "/a /b 200x",
+ "a /b",
+ "/caf%ZZ /c",
+ "/a%2Fb /c",
+ "/%3Aname /c",
+ "/a b",
+ "/a //evil.example",
+ `/a /\evil.example`,
+ "/a/*/b /c",
+ "/a* /c",
+ "/:splat /c",
+ "/:a/:a /c",
+ "/:a-b /c",
+ "/a//b /c",
+ "/a?x=1 /c",
+ "/a/../b /c",
+ "https://www.example.com:8443/ /c",
+ "/a https://example.org/ 200",
+ "/a https://example.org/ 404",
+ "/a ftp://example.org/",
+ "/a https:///nohost",
+ } {
+ rs, err := parseRedirects([]byte(line + "\n"))
+ if err == nil {
+ t.Errorf("parseRedirects(%q) = %+v, want an error", line, rs.rules)
+ continue
+ }
+ if !strings.Contains(err.Error(), "line 1:") {
+ t.Errorf("parseRedirects(%q) error %q does not name the line", line, err)
+ }
+ if len(rs.rules) != 0 {
+ t.Errorf("parseRedirects(%q) kept %d rules", line, len(rs.rules))
+ }
+ }
+}
+
+func TestParseRedirectsKeepsTheGoodLinesAroundABadOne(t *testing.T) {
+ rs, err := parseRedirects([]byte("/a /b\n/c /d 999\n/e /f\n"))
+ if err == nil || !strings.Contains(err.Error(), "line 2:") {
+ t.Fatalf("err = %v, want one naming line 2", err)
+ }
+ if len(rs.rules) != 2 || rs.rules[1].line != 3 {
+ t.Fatalf("rules = %+v, want lines 1 and 3", rs.rules)
+ }
+}
+
+func TestParseRedirectsLimits(t *testing.T) {
+ if _, err := parseRedirects([]byte(strings.Repeat("#", redirectsMaxBytes+1))); err == nil {
+ t.Fatal("an oversized file parsed")
+ }
+ var b strings.Builder
+ for range redirectsMaxRules + 1 {
+ b.WriteString("/a /b\n")
+ }
+ rs, err := parseRedirects([]byte(b.String()))
+ if err == nil || len(rs.rules) != redirectsMaxRules {
+ t.Fatalf("%d rules, err %v: want %d rules and an error", len(rs.rules), err, redirectsMaxRules)
+ }
+}
+
+func TestPackDirRefusesABadRedirectsFile(t *testing.T) {
+ dir := writeTree(t, map[string]string{
+ "index.html": "home",
+ "_redirects": "/ok /fine\n/a /b 418\n",
+ })
+ _, err := PackDir(dir)
+ if err == nil || !strings.Contains(err.Error(), "_redirects") || !strings.Contains(err.Error(), "line 2:") {
+ t.Fatalf("PackDir = %v, want a refusal naming _redirects line 2", err)
+ }
+ // Only the root file is the rules file; one deeper is content.
+ if _, err := PackDir(writeTree(t, map[string]string{"sub/_redirects": "not rules"})); err != nil {
+ t.Fatalf("PackDir with a nested _redirects: %v", err)
+ }
+}
+
+// redirectFixture ships a site with a _redirects file behind a static
+// handler, and returns a client that does not follow redirects.
+func redirectFixture(t *testing.T, rules string) (*httptest.Server, *http.Client) {
+ t.Helper()
+ store := release.NewDirStore(filepath.Join(t.TempDir(), "store"))
+ priv, pub := testKeys(t)
+ shipSite(t, store, "site", "v1", map[string]string{
+ "index.html": "home",
+ "old.html": "old page",
+ "forced.html": "forced page",
+ "style.css": "body{}",
+ "about/index.html": "about",
+ "app/index.html": "the app",
+ "docs/page.html": "a doc",
+ "404.html": "<!doctype html><p>custom not found",
+ "_redirects": rules,
+ })
+ if err := release.Promote(context.Background(), store, priv, release.OpsAccount, "site", "canary/x", "v1"); err != nil {
+ t.Fatalf("Promote: %v", err)
+ }
+ rt := registry.Route{Host: "site.example.com", Kind: "static", App: "site", Channel: "canary/x", Status: "ready"}
+ h := newStaticHandler(rt, store, pub, filepath.Join(t.TempDir(), "cache"))
+ srv := httptest.NewServer(h)
+ t.Cleanup(srv.Close)
+ client := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
+ return srv, client
+}
+
+type redirectResult struct {
+ code int
+ location string
+ body string
+ resp *http.Response
+}
+
+func fetch(t *testing.T, c *http.Client, method, base, host, path string) redirectResult {
+ t.Helper()
+ req, err := http.NewRequest(method, base+path, nil)
+ if err != nil {
+ t.Fatalf("new request: %v", err)
+ }
+ req.Host = host
+ resp, err := c.Do(req)
+ if err != nil {
+ t.Fatalf("%s %s: %v", method, path, err)
+ }
+ defer resp.Body.Close()
+ b, _ := io.ReadAll(resp.Body)
+ return redirectResult{resp.StatusCode, resp.Header.Get("Location"), string(b), resp}
+}
+
+const siteRules = `
+/old.html /new.html 301
+/forced.html /elsewhere 302!
+/blog/* /posts/:splat 301
+/docs/:slug /guide/:slug?from=:slug 302
+/app/* /app/index.html 200
+https://www.example.com/* https://site.example.com/:splat 301!
+/loop/ /loop 301
+/ext https://example.org/x?y=1 307
+/empty/* /:splat/tail 301
+/frag /target#section 302
+/slash/* /slash/:splat/ 301!
+/caf%C3%A9 /cafe 301
+/files/:n /docs/:n.html 200
+/missing-rewrite /no/such/file 200
+/* /404.html 404
+`
+
+func TestStaticRedirects(t *testing.T) {
+ srv, c := redirectFixture(t, siteRules)
+ const host = "site.example.com"
+ for _, tc := range []struct {
+ host, path string
+ code int
+ location string // checked when code is a redirect
+ body string // checked when code is not
+ }{
+ // A file at the path shadows a rule; "!" overrides that.
+ {host, "/old.html", 200, "", "old page"},
+ {host, "/forced.html", 302, "/elsewhere", ""},
+ // Splats, placeholders, query strings.
+ {host, "/blog/2024/hello", 301, "/posts/2024/hello", ""},
+ {host, "/blog/2024/hello?x=1", 301, "/posts/2024/hello?x=1", ""},
+ {host, "/blog/2024/", 301, "/posts/2024/", ""},
+ {host, "/blog", 301, "/posts/", ""},
+ {host, "/docs/a%20b", 302, "/guide/a%20b?from=a+b", ""},
+ // Paths match decoded, as files are found: %2F separates segments.
+ {host, "/docs/a%2Fb", 404, "", "<!doctype html><p>custom not found"},
+ {host, "/ext?a=b", 307, "https://example.org/x?y=1", ""},
+ {host, "/frag?q=1", 302, "/target?q=1#section", ""},
+ // An existing file outside the rules is untouched.
+ {host, "/docs/page.html", 200, "", "a doc"},
+ // A directory with an index exists; a path under it is rewritten.
+ {host, "/app/", 200, "", "the app"},
+ {host, "/app/settings/profile", 200, "", "the app"},
+ // A redirect to the URL requested is skipped, not looped.
+ {host, "/loop/", 301, "/loop", ""},
+ {host, "/loop", 404, "", "<!doctype html><p>custom not found"},
+ // An empty capture never yields a protocol-relative Location.
+ {host, "/empty/", 301, "/tail", ""},
+ // A trailing-slash rule stops once the slash is there.
+ {host, "/slash/x", 301, "/slash/x/", ""},
+ {host, "/slash/x/", 404, "", "<!doctype html><p>custom not found"},
+ // A percent-encoded source matches the decoded request.
+ {host, "/caf%C3%A9", 301, "/cafe", ""},
+ // A "#" in a capture is part of the file name, not its end.
+ {host, "/files/page", 200, "", "a doc"},
+ {host, "/files/page.html%23", 404, "", "404 page not found\n"},
+ // The host rule matches its host only, and keeps the trailing slash.
+ {"www.example.com", "/about/", 301, "https://site.example.com/about/", ""},
+ {"WWW.Example.com.", "/style.css", 301, "https://site.example.com/style.css", ""},
+ {host, "/about/", 200, "", "about"},
+ // A rewrite to nothing is a plain 404; everything else hits /*.
+ {host, "/missing-rewrite", 404, "", "404 page not found\n"},
+ {host, "/nope", 404, "", "<!doctype html><p>custom not found"},
+ // The rules file is never served.
+ {host, "/_redirects", 404, "", "<!doctype html><p>custom not found"},
+ } {
+ got := fetch(t, c, "GET", srv.URL, tc.host, tc.path)
+ if got.code != tc.code {
+ t.Errorf("GET %s%s = %d (Location %q, body %q), want %d", tc.host, tc.path, got.code, got.location, got.body, tc.code)
+ continue
+ }
+ if tc.code >= 300 && tc.code < 400 {
+ if got.location != tc.location {
+ t.Errorf("GET %s%s Location = %q, want %q", tc.host, tc.path, got.location, tc.location)
+ }
+ } else if got.body != tc.body {
+ t.Errorf("GET %s%s body = %q, want %q", tc.host, tc.path, got.body, tc.body)
+ }
+ if v := got.resp.Header.Get("X-Carlos-Version"); v != "v1" {
+ t.Errorf("GET %s%s X-Carlos-Version = %q, want v1", tc.host, tc.path, v)
+ }
+ if cc := got.resp.Header.Get("Cache-Control"); cc != "no-cache" && tc.code != 404 {
+ t.Errorf("GET %s%s Cache-Control = %q, want no-cache", tc.host, tc.path, cc)
+ }
+ }
+
+ // The not-found page is typed and carries no validator.
+ got := fetch(t, c, "GET", srv.URL, host, "/nope")
+ if ct := got.resp.Header.Get("Content-Type"); !strings.HasPrefix(ct, "text/html") {
+ t.Errorf("404 page Content-Type = %q, want text/html", ct)
+ }
+ if et := got.resp.Header.Get("ETag"); et != "" {
+ t.Errorf("404 page ETag = %q, want none", et)
+ }
+ if head := fetch(t, c, "HEAD", srv.URL, host, "/nope"); head.code != 404 || head.body != "" {
+ t.Errorf("HEAD /nope = %d %q, want 404 with no body", head.code, head.body)
+ }
+}
+
+func TestStaticSPAFallbackLeavesAssetsAlone(t *testing.T) {
+ srv, c := redirectFixture(t, "/* /index.html 200\n")
+ const host = "site.example.com"
+ for path, want := range map[string]string{
+ "/style.css": "body{}",
+ "/about/": "about",
+ "/some/client/route": "home",
+ "/_redirects": "home",
+ } {
+ if got := fetch(t, c, "GET", srv.URL, host, path); got.code != 200 || got.body != want {
+ t.Errorf("GET %s = %d %q, want 200 %q", path, got.code, got.body, want)
+ }
+ }
+ // A rewritten page revalidates like the file it serves.
+ first := fetch(t, c, "GET", srv.URL, host, "/some/route")
+ tag := first.resp.Header.Get("ETag")
+ if tag == "" {
+ t.Fatal("rewritten page has no ETag")
+ }
+ req, _ := http.NewRequest("GET", srv.URL+"/other/route", nil)
+ req.Host = host
+ req.Header.Set("If-None-Match", tag)
+ resp, err := c.Do(req)
+ if err != nil {
+ t.Fatal(err)
+ }
+ resp.Body.Close()
+ if resp.StatusCode != http.StatusNotModified {
+ t.Fatalf("conditional GET of a rewritten page = %d, want 304", resp.StatusCode)
+ }
+}
+
+func TestStaticWithoutRedirectsIsUnchanged(t *testing.T) {
+ _, _, _, srv := staticFixture(t)
+ c := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
+ if got := fetch(t, c, "GET", srv.URL, "site.example.com", "/missing"); got.code != 404 || got.body != "404 page not found\n" {
+ t.Fatalf("GET /missing = %d %q, want the plain 404", got.code, got.body)
+ }
+ if got := fetch(t, c, "GET", srv.URL, "site.example.com", "/docs/page.html/index.html"); got.code != 404 {
+ t.Fatalf("GET under a file = %d, want 404", got.code)
+ }
+}
diff --git a/internal/edge/static.go b/internal/edge/static.go
index a61779f77..2f2fd0907 100644
--- a/internal/edge/static.go
+++ b/internal/edge/static.go
@@ -28,6 +28,7 @@ import (
"path"
"path/filepath"
"sort"
+ "strconv"
"strings"
"sync"
"sync/atomic"
@@ -96,6 +97,14 @@ func PackDir(dir string) ([]byte, error) {
if _, err := tw.Write(b); err != nil {
return nil, err
}
+ if name == redirectsFile {
+ // Refused here rather than skipped at the edge, so a rule the
+ // edge cannot use is an error the shipper sees, not a redirect
+ // that silently never happens.
+ if _, err := parseRedirects(b); err != nil {
+ return nil, fmt.Errorf("pack %s: %s: %w", dir, name, err)
+ }
+ }
}
if err := tw.Close(); err != nil {
return nil, err
@@ -123,10 +132,11 @@ type staticSite struct {
recheck time.Duration
mu sync.Mutex
- version string // currently served version; "" until first resolve
- checkedAt time.Time // when the pointer was last read
- digest string // the served tree's manifest digest
- name string // the served tree's directory: siteDirName(version, digest)
+ version string // currently served version; "" until first resolve
+ checkedAt time.Time // when the pointer was last read
+ digest string // the served tree's manifest digest
+ name string // the served tree's directory: siteDirName(version, digest)
+ redirects *redirectRules // the served tree's _redirects; nil when it has none
// ratchet is #253's per-box ratchet for this process's pointer reads
// (Edge.staticRatchet); nil admits every verified pointer.
@@ -152,7 +162,7 @@ func newStaticHandler(rt registry.Route, store release.Store, pub ed25519.Public
}
func (s *staticSite) ServeHTTP(w http.ResponseWriter, r *http.Request) {
- dir, version, err := s.currentDir(r.Context())
+ tree, err := s.currentDir(r.Context())
if err != nil {
// Nothing has ever resolved here, so there is no version to name:
// this 502 stays unstamped rather than claiming a build it isn't
@@ -172,7 +182,7 @@ func (s *staticSite) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// stamps what is actually served, never the pointer's unreachable head.
// Set before any body is written, so it lands on the tree's own 404s too
// (the handler serving that tree IS a version).
- w.Header().Set("X-Carlos-Version", version)
+ w.Header().Set("X-Carlos-Version", tree.version)
// A page out of the tree is revalidated before it is reused, and so is
// the tree's own 404. With Last-Modified alone a browser GUESSES a
// freshness lifetime — typically a tenth of the file's age — and reuses
@@ -185,7 +195,15 @@ func (s *staticSite) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// from the errors it writes itself, a 416 or a 400: neither is a status a
// browser caches on a guess.)
w.Header().Set("Cache-Control", "no-cache")
- s.serveTree(dir, w, r)
+ s.serveTree(tree, w, r)
+}
+
+// servedTree is what one request is answered from: an extracted tree, the
+// version it holds, and the rules its _redirects declared. They travel
+// together so a request never pairs one version's files with another's rules.
+type servedTree struct {
+ dir, version string
+ redirects *redirectRules
}
// dir is where the tree named name lives once extracted.
@@ -216,7 +234,8 @@ func (s *staticSite) appDir() string {
// afterwards so ServeHTTP can stamp X-Carlos-Version without re-taking mu (and
// without the window in between, where a concurrent resolve could swap
// s.version and make the header describe a different tree than the one served).
-func (s *staticSite) currentDir(ctx context.Context) (dir, version string, err error) {
+// The tree's redirect rules come back with it for the same reason.
+func (s *staticSite) currentDir(ctx context.Context) (servedTree, error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.version != "" && s.recheck > 0 && time.Since(s.checkedAt) < s.recheck {
@@ -224,7 +243,7 @@ func (s *staticSite) currentDir(ctx context.Context) (dir, version string, err e
// version, and stamping that version is honest by construction — the
// header names what the bytes are, not what the pointer may have
// advanced to since the last check.
- return s.dir(s.name), s.version, nil
+ return s.served(), nil
}
cp, err := release.Channel(ctx, s.store, s.pub, s.acct, s.app, s.channel)
if err != nil {
@@ -242,22 +261,48 @@ func (s *staticSite) currentDir(ctx context.Context) (dir, version string, err e
return s.stale(fmt.Errorf("swap current -> %s: %w", name, err))
}
s.version, s.digest, s.name = cp.Version, digest, name
+ s.redirects = s.loadRedirects(s.dir(name), cp.Version)
}
s.checkedAt = time.Now()
- return s.dir(s.name), s.version, nil
+ return s.served(), nil
+}
+
+// served is the tree currently being served. The caller holds mu.
+func (s *staticSite) served() servedTree {
+ return servedTree{dir: s.dir(s.name), version: s.version, redirects: s.redirects}
+}
+
+// loadRedirects parses the _redirects at the root of an extracted tree, once
+// per version swap. A tree without one has no rules. A line that does not
+// parse is logged and skipped: ship refuses such a tree (PackDir), so only a
+// tree packed some other way gets here, and its remaining rules still apply.
+func (s *staticSite) loadRedirects(dir, version string) *redirectRules {
+ data, err := os.ReadFile(filepath.Join(dir, redirectsFile))
+ if errors.Is(err, fs.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ log.Printf("edge: static %s/%s: %s %s: %v — serving without its rules", s.app, s.channel, version, redirectsFile, err)
+ return nil
+ }
+ rules, err := parseRedirects(data)
+ if err != nil {
+ log.Printf("edge: static %s/%s: %s %s: %s", s.app, s.channel, version, redirectsFile, strings.ReplaceAll(err.Error(), "\n", "; "))
+ }
+ return rules
}
// stale is the degraded path: keep serving the version we already verified
// and extracted — and name it, so the response header describes the CACHED
// tree rather than the head this handler could not reach — or fail if there
// has never been one.
-func (s *staticSite) stale(err error) (string, string, error) {
+func (s *staticSite) stale(err error) (servedTree, error) {
if s.version == "" {
- return "", "", err
+ return servedTree{}, err
}
log.Printf("edge: static %s/%s: %v — still serving cached %s", s.app, s.channel, err, s.version)
s.checkedAt = time.Now() // back off: retry next recheck window, not next request
- return s.dir(s.name), s.version, nil
+ return s.served(), nil
}
// ensureExtracted makes the version's verified tree exist and returns its
@@ -398,7 +443,9 @@ func extractSiteTar(data []byte, dst string) error {
// serveTree answers one request from an extracted version dir:
// http.FileServer semantics with index.html for directories and a clean 404
-// (never a directory listing) when neither the file nor an index exists.
+// (never a directory listing) when neither the file nor an index exists —
+// after the tree's _redirects (redirects.go) has had its say. A rule applies
+// only where no file answers the path, unless it is forced.
//
// It serves the file itself rather than through http.ServeFile, because
// ServeFile offers the file's mtime as Last-Modified and answers
@@ -412,7 +459,7 @@ func extractSiteTar(data []byte, dst string) error {
// The validator is the file's content (tagCache). What ServeFile did besides
// is kept by hand: the refusal of a ".." element, and the redirect off a
// trailing /index.html onto its directory.
-func (s *staticSite) serveTree(dir string, w http.ResponseWriter, r *http.Request) {
+func (s *staticSite) serveTree(tree servedTree, w http.ResponseWriter, r *http.Request) {
if containsDotDot(r.URL.Path) {
// The Clean below would resolve it harmlessly inside the tree; this
// keeps the answer ServeFile always gave such a path.
@@ -421,27 +468,54 @@ func (s *staticSite) serveTree(dir string, w http.ResponseWriter, r *http.Reques
}
// Clean rooted at "/" so no request path can climb out of the tree.
p := path.Clean("/" + r.URL.Path)
- fp := filepath.Join(dir, filepath.FromSlash(p))
- fi, err := os.Stat(fp)
- if err == nil && fi.IsDir() {
- fp = filepath.Join(fp, "index.html")
- fi, err = os.Stat(fp)
- }
- if err != nil || fi.IsDir() {
- http.NotFound(w, r)
- return
- }
- if strings.HasSuffix(r.URL.Path, "/index.html") {
- // Relative, as ServeFile wrote it, so it survives any prefix the
- // request arrived under.
- loc := "./"
- if q := r.URL.RawQuery; q != "" {
- loc += "?" + q
+ status, rewritten := http.StatusOK, false
+ for {
+ fp := filepath.Join(tree.dir, filepath.FromSlash(p))
+ fi, err := os.Stat(fp)
+ if err == nil && fi.IsDir() {
+ fp = filepath.Join(fp, "index.html")
+ fi, err = os.Stat(fp)
}
- w.Header().Set("Location", loc)
- w.WriteHeader(http.StatusMovedPermanently)
+ // The _redirects file is configuration and answers nothing.
+ found := err == nil && !fi.IsDir() && p != "/"+redirectsFile
+ if !rewritten {
+ if act, ok := tree.redirects.resolve(requestHost(r.Host), p, r.URL.Path, r.URL.RawQuery, found); ok {
+ if act.location != "" {
+ http.Redirect(w, r, act.location, act.status)
+ return
+ }
+ // A rewrite serves another file of the tree at this URL. It is
+ // looked up once: rules are not consulted a second time, and a
+ // destination with no file is a plain 404.
+ p, status, rewritten = act.file, act.status, true
+ continue
+ }
+ }
+ if !found {
+ http.NotFound(w, r)
+ return
+ }
+ if !rewritten && strings.HasSuffix(r.URL.Path, "/index.html") {
+ // Relative, as ServeFile wrote it, so it survives any prefix the
+ // request arrived under.
+ loc := "./"
+ if q := r.URL.RawQuery; q != "" {
+ loc += "?" + q
+ }
+ w.Header().Set("Location", loc)
+ w.WriteHeader(http.StatusMovedPermanently)
+ return
+ }
+ s.serveFile(fp, p, fi, status, w, r)
return
}
+}
+
+// serveFile sends the tree file at fp, whose tree path is p (for logs). A 200
+// gets a content tag and net/http's conditional and range handling. Any other
+// status — a _redirects 404 page — is the file's bytes under that status,
+// with no validator: ServeContent knows only how to succeed.
+func (s *staticSite) serveFile(fp, p string, fi os.FileInfo, status int, w http.ResponseWriter, r *http.Request) {
f, err := os.Open(fp)
if err != nil {
// ServeFile's mapping, kept: a file that vanished since the Stat is
@@ -459,6 +533,26 @@ func (s *staticSite) serveTree(dir string, w http.ResponseWriter, r *http.Reques
return
}
defer f.Close()
+ if status != http.StatusOK {
+ ctype := mime.TypeByExtension(filepath.Ext(fp))
+ if ctype == "" {
+ var head [512]byte
+ n, _ := io.ReadFull(f, head[:])
+ ctype = http.DetectContentType(head[:n])
+ if _, err := f.Seek(0, io.SeekStart); err != nil {
+ log.Printf("edge: static %s/%s: rewind %s: %v", s.app, s.channel, p, err)
+ http.Error(w, "500 Internal Server Error", http.StatusInternalServerError)
+ return
+ }
+ }
+ w.Header().Set("Content-Type", ctype)
+ w.Header().Set("Content-Length", strconv.FormatInt(fi.Size(), 10))
+ w.WriteHeader(status)
+ if r.Method != http.MethodHead {
+ io.Copy(w, f)
+ }
+ return
+ }
// Stat the OPEN file: its identity is what a remembered tag is checked
// against, and it is the file whose bytes are about to be sent.
if ofi, err := f.Stat(); err == nil && !ofi.IsDir() {
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go
index 5b5d181c5..a38913fe7 100644
--- a/internal/provision/plan14_root_reads_test.go
+++ b/internal/provision/plan14_root_reads_test.go
@@ -1408,6 +1408,21 @@ func TestPlan14RootReadCensus(t *testing.T) {
if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-db-region-forward-copy.tsv"), &inv, true); err != nil {
t.Fatal(err)
}
+ // Census overlay static-redirects (static sites honour a _redirects
+ // file), loaded last, after db-region-forward-copy. Proposed; ruling
+ // asked of the census owner on #280. PackDir and staticSite.serveTree are
+ // re-frozen; staticSite.serveFile takes serveTree's Open, and
+ // staticSite.loadRedirects reads the tree's _redirects once per version
+ // swap. Every read stays inside the extracted, verified tree. Frozen
+ // declarations are added, so the seal moves; plans-3a-creation-caps's
+ // (unchanged by db-region-forward-copy) is checked first.
+ if errs := checkRootReadLegacy(inv, seal); len(errs) != 0 {
+ t.Fatal(errs)
+ }
+ if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-static-redirects.tsv"), &inv, true); err != nil {
+ t.Fatal(err)
+ }
+ seal = "65857c29501dc7676f52921bea79d08217542ecb1d775c67d20a3460c851d831"
errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
errs = append(errs, checkRootReadDeclarations(got, inv)...)
if len(errs) > 0 {
diff --git a/internal/provision/testdata/plan14-root-reads-static-redirects.tsv b/internal/provision/testdata/plan14-root-reads-static-redirects.tsv
new file mode 100644
index 000000000..3ee529d89
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-static-redirects.tsv
@@ -0,0 +1,11 @@
+# Census overlay static-redirects (static sites honour a _redirects file: redirects, rewrites and a not-found page), loaded last, after db-region-forward-copy. PROPOSED-static-redirects; the ruling is asked of the census owner on #280.
+# PackDir (the CLI's site packer) and staticSite.serveTree (the edge's static handler) are re-frozen; serveTree's Open moves into the new staticSite.serveFile, and the new staticSite.loadRedirects reads the tree's _redirects once per version swap. Every read stays inside the extracted, digest-verified tree; none is a root decision. Frozen declarations are added, so the seal moves; the internal/edge envelope is re-pinned.
+@legacy 4c0e4dfcd4f19d14eaf317510eb7968c4388507a61dd1504b62d7ece172b80a9 internal/edge/static.go PackDir legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;rev3=PROPOSED-static-redirects;identity=the carlos CLI (carlos ship and carlos deploy --kind static, on the member's own machine);basis=also parses the root _redirects file it has already read (the same ReadFile) and refuses the pack when a line does not parse; no read added, offsets unchanged file:ReadFile@626,file:WalkDir@71 16bf7592a19c80d6447dc6a9b3fa5dd65e881dbe3b0c6efd8b510efb3cf6c120
+@legacy ac8b1bc328c04f742fd0c2245adb9258ea689062f83226017f18d638bb795aff internal/edge/static.go staticSite.serveTree legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=the same index-aware Stat lookup inside the extracted tree, now run at most twice (the request path, then once for a _redirects rewrite destination cleaned rooted inside the same tree); the Open moves to serveFile; the _redirects file itself is never answered; no root decision file:Stat@545,file:Stat@642 c0e1003c7eb524445bada46e0fb35f456ae82273db6a58d8cc3163f5017cc065
+@legacy fa5fa8fd610a49985faa232d832a492c6d02e6365394e11e022a83a7377bea59 internal/edge/static.go staticSite.serveFile legacy accepted-legacy accepted-legacy;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=root-input; serveTree's former Open and open-file Stat of the file serveTree resolved inside the extracted, digest-verified tree, unchanged, plus a non-200 branch that copies the same open file under a _redirects 404 status; no root decision file:Open@126,file:Stat@1503 4f2f2217d586cbb588f3a96af125f064e6818911dc9e7e92f70db3e0aaa9e498
+@legacy 5c2e4f1044884dc6400ccacc618ffaa4746507b27f57bc4eebda54f989018898 internal/edge/static.go staticSite.loadRedirects legacy accepted-legacy accepted-legacy;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=root-input; reads the _redirects file at the root of the extracted, digest-verified tree once per version swap, under the site's mutex, into immutable per-version rules; an unreadable or bad file only loses rules; no root decision file:ReadFile@87 31999f705b500658bc4ef627b978a4e4f2917321056856f167ed00751c9de303
+internal/edge/static.go PackDir legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;rev3=PROPOSED-static-redirects;identity=the carlos CLI (carlos ship and carlos deploy --kind static, on the member's own machine);basis=also parses the root _redirects file it has already read (the same ReadFile) and refuses the pack when a line does not parse; no read added, offsets unchanged file:ReadFile@626,file:WalkDir@71 16bf7592a19c80d6447dc6a9b3fa5dd65e881dbe3b0c6efd8b510efb3cf6c120
+internal/edge/static.go staticSite.serveTree legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=the same index-aware Stat lookup inside the extracted tree, now run at most twice (the request path, then once for a _redirects rewrite destination cleaned rooted inside the same tree); the Open moves to serveFile; the _redirects file itself is never answered; no root decision file:Stat@545,file:Stat@642 c0e1003c7eb524445bada46e0fb35f456ae82273db6a58d8cc3163f5017cc065
+internal/edge/static.go staticSite.serveFile legacy accepted-legacy accepted-legacy;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=root-input; serveTree's former Open and open-file Stat of the file serveTree resolved inside the extracted, digest-verified tree, unchanged, plus a non-200 branch that copies the same open file under a _redirects 404 status; no root decision file:Open@126,file:Stat@1503 4f2f2217d586cbb588f3a96af125f064e6818911dc9e7e92f70db3e0aaa9e498
+internal/edge/static.go staticSite.loadRedirects legacy accepted-legacy accepted-legacy;rev3=PROPOSED-static-redirects;identity=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos);basis=root-input; reads the _redirects file at the root of the extracted, digest-verified tree once per version swap, under the site's mutex, into immutable per-version rules; an unreadable or bad file only loses rules; no root decision file:ReadFile@87 31999f705b500658bc4ef627b978a4e4f2917321056856f167ed00751c9de303
+@coverage internal/edge f9465bb2791d0fbbe9743431d3da5d24ab4397d6f1f0ec2c2533d3cc091a7487