rastrillo / fiesta Public

Write a _headers file that caches releases for a year and /M/ for minutes

Every file fiesta.rastrillo.org serves goes out with CARLOS's default
Cache-Control: no-cache, so every page on every site that links
/1.0.0/fiesta.css revalidates it on every view, for bytes that cannot
change. CARLOS v0.31.1 reads a _headers file at the root of a static
tree, and site.Build now writes one beside _redirects, from the same
inputs, so cutting a release adds its own rule.

/M/* gets five minutes, not a year: the edge matches the address the
browser asked for, before the /M/ rewrite, and /1/ moves to each new
1.x. Each release's /M.y.z/* gets a year and immutable. Both carry
Access-Control-Allow-Origin: * and Cross-Origin-Resource-Policy:
cross-origin, so other sites can fetch the strings, load modules and
pin files by hash. The gallery's hashed assets get a year and
immutable alone; only the gallery loads them. Everything else keeps
no-cache, so a deploy shows on the next load.

The hashed assets are found by gallery.Fingerprinted, which lists a
hashed name only when its plain twin is beside it with the same bytes.
A regexp on the name was rejected: a name that merely looks hashed
could take other bytes later, and a browser told it never changes
would keep the old ones. www.Rules hands both the redirects and these
addresses to site.Build, which now takes a Rules value in place of the
variadic redirects, so the three callers cannot keep one and drop the
other.

Build refuses what carlos ship would refuse, so it fails on the
branch and not on the day the site ships: a header outside the five
the edge allows, or a value it refuses (headers are constants, so
this fires only when someone edits them), a line over 2,048
characters, and more than 90 paths, ten short of the edge's 100, with
what to do. An immutable address must be a page, spelled so the edge
reads it as that page's path. A page named _headers is refused, as
_redirects is.

With a path per major and per release, _headers stops a site at 45
majors, so the _redirects test that built 2,000 majors now reaches
2,000 rules through the pages' rules instead.

internal/www's tests read the built file with a mirror of the edge's
parser and hold it to exactly its rules, immutability exactly on
releases and hashed assets, every rule matching a built file, and the
edge's limits.

Red: each new test failed by assertion against a Build that wrote no
_headers and a Fingerprinted that listed nothing. Mutations, each red
and reverted: releases cached five minutes; /M/ cached a year; plain
assets listed; any hashed-looking name listed; www.Rules dropping the
immutable addresses; the stop at 100; any origin accepted; any header
name accepted; no line limit; an immutable address not checked
against the pages; cross-origin headers on the hashed assets.
Human

Prompt

Add a CARLOS `_headers` file to Fiesta's built site, on branch fiesta-1-1, so that 1.1 ships with it.

Background. CARLOS v0.31.1, live on the flagship, lets a static site ship a `_headers` file at the root of its tree, beside `_redirects`. Rules match the path the browser asked for, before any rewrite; every matching rule applies, and the last one wins per header. Only five headers are allowed: Cache-Control; Access-Control-Allow-Origin, `*` only; Access-Control-Expose-Headers; Cross-Origin-Resource-Policy; Timing-Allow-Origin. A file can have at most 100 paths and 2,048 characters a line. A matching Cache-Control replaces the default `no-cache`.

What to generate. internal/site/site.go writes `_redirects` today; generate `_headers` the same way, from the releases that are built, so a new release adds its own rule.
1. Each published version, `/<x.y.z>/*`: `Cache-Control: public, max-age=31536000, immutable`, `Access-Control-Allow-Origin: *` and `Cross-Origin-Resource-Policy: cross-origin`.
2. `/1/*`, and every other major the site serves the same way: `Cache-Control: public, max-age=300`, plus the same two cross-origin headers. `/1/` is rewritten to the newest 1.x, so it must stay short-lived.
3. The gallery's fingerprinted assets, the files named `name.<16 hex>.ext` under /design-system/assets/: `Cache-Control: public, max-age=31536000, immutable`. List each path exactly. They need no CORS.
4. Everything else (HTML, `/static/*`, the unfingerprinted copies) gets no rule, so the platform's `no-cache` default stays.
Order general rules before specific ones. Write short `#` comments in the file that say why.

Limits. The builder must fail the build, with a message, if the file would break CARLOS's limits. It must also fail if it would set a header outside the allowed five. Stop at 90 paths with a message saying what to do.

Tests, red first. The built `_headers` has exactly the rules above. Every fingerprinted file in the built tree has an immutable rule, and nothing unfingerprinted does. Every rule's path matches at least one built file. The build refuses an oversized file or a disallowed header.

If an internal/www test serves the built site the way the edge does, decide whether it should apply `_headers` too. If it can be done cheaply, add an assertion that a request for `/1/fiesta.css` would get `max-age=300` and not immutable, since rules match before the rewrite.

From this commit's own message. The prompt ledger has no entries for it — which is what a squashed, rebased or amended commit looks like, and what every commit looks like in a repo without a ledger.

Summary

site.Build writes _headers from the releases and the gallery's hashed assets, refusing what CARLOS's edge would refuse.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude agent, operated by paul@keymail.dev b1e5fd0f1bb50d6bf19efc350fa0844ad1510760 parent d023e75
14 files changed, +1000 −65
  • Makefile +2 −1
  • cmd/fiesta/main.go +4 −3
  • cmd/fiesta/release_test.go +1 −1
  • hack/test-weights.txt +1 −0
  • internal/gallery/hashed.go +25 −0
  • internal/gallery/hashed_test.go +38 −0
  • internal/site/export_test.go +8 −0
  • internal/site/site.go +263 −18
  • internal/site/site_test.go +246 −32
  • internal/www/headers_test.go +380 −0
  • internal/www/serve_test.go +1 −1
  • internal/www/www.go +29 −7
  • internal/www/www_test.go +1 −1
  • releases_test.go +1 −1
diff --git a/Makefile b/Makefile
index 2c0e41a..8974d27 100644
--- a/Makefile
+++ b/Makefile
@@ -257,7 +257,8 @@ heights-sweep: | $(BIN)/tmp
# site assembles fiesta.rastrillo.org into .build/site/public from the
# committed releases and the site source: each release under its exact
# version, _redirects sending /M/* to the newest M.y.z and the design
-# system's former addresses to their pages, and at the root the pages
+# system's former addresses to their pages, _headers caching each
+# release for a year and /M/ for five minutes, and at the root the pages
# internal/www assembles from site/, the landing page, the design
# system under /design-system/ and site/static/. The tree is
# .build/site/public; the command's own .fiesta stamp sits above it in
diff --git a/cmd/fiesta/main.go b/cmd/fiesta/main.go
index 4777f7a..841ed28 100644
--- a/cmd/fiesta/main.go
+++ b/cmd/fiesta/main.go
@@ -34,8 +34,9 @@
// assembles the site from the releases and the site source, exactly
// what internal/site.Build returns, into DIR/public/: each release
// under its exact version, _redirects sending /M/* to the newest
-// M.y.z and the design system's former addresses to their pages, and
-// the pages at the root. The pages are what internal/www assembles
+// M.y.z and the design system's former addresses to their pages,
+// _headers caching each release for a year and /M/ for five minutes,
+// and the pages at the root. The pages are what internal/www assembles
// from the site source: the landing page, the design system under
// /design-system/, and the source's static/ files. DIR/public/ is the
// tree to ship, and the stamp sits above it in DIR, so nothing hidden
@@ -214,7 +215,7 @@ func writeSite(releases, siteDir, out string) error {
if err != nil {
return err
}
- files, err := site.Build(os.DirFS(releases), www.FS(pages), www.Redirects(pages)...)
+ files, err := site.Build(os.DirFS(releases), www.FS(pages), www.Rules(pages))
if err != nil {
return err
}
diff --git a/cmd/fiesta/release_test.go b/cmd/fiesta/release_test.go
index 58d5542..7140402 100644
--- a/cmd/fiesta/release_test.go
+++ b/cmd/fiesta/release_test.go
@@ -413,7 +413,7 @@ func TestSiteWritesWhatBuildReturns(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- want, err := site.Build(os.DirFS(releases), www.FS(pages), www.Redirects(pages)...)
+ want, err := site.Build(os.DirFS(releases), www.FS(pages), www.Rules(pages))
if err != nil {
t.Fatal(err)
}
diff --git a/hack/test-weights.txt b/hack/test-weights.txt
index 940171a..6cd0e5c 100644
--- a/hack/test-weights.txt
+++ b/hack/test-weights.txt
@@ -255,4 +255,5 @@ test internal/www TestPagesHasARootAndTheDesignSystem 5
test internal/www TestStaticFilesAreCopied 5
test internal/www TestTheDesignSystemIsMounted 5
test internal/www TestTheRedirectsFileIsWithinCarlosLimits 5
+test internal/www TestTheSitesHeaders 5
test internal/www TestAHiddenSiteFileIsRefused 4
diff --git a/internal/gallery/hashed.go b/internal/gallery/hashed.go
index 98111c2..4ad196f 100644
--- a/internal/gallery/hashed.go
+++ b/internal/gallery/hashed.go
@@ -130,3 +130,28 @@ func (s *site) asset(name string) string {
// Asset is the URL of one of the gallery's assets, by its plain name.
func (p page) Asset(name string) string { return p.s.asset(name) }
+
+// Fingerprinted are the addresses, for a tree rendered for mount, of
+// every asset written under its hashed name, sorted: each hashed twin
+// the tree holds beside its plain name, with the same bytes. The name
+// is worked out from the plain file's bytes, as putAsset works it out,
+// so a name that only looks hashed, with no plain twin or with hex
+// that is not its bytes' hash, is never listed: other bytes could be
+// written under it, and a browser told it never changes would keep
+// the old ones.
+func Fingerprinted(mount string, tree map[string][]byte) []string {
+ m := cleanMount(mount)
+ var out []string
+ for name, body := range tree {
+ plain, ok := strings.CutPrefix(name, "assets/")
+ if !ok {
+ continue
+ }
+ hashed := "assets/" + hashedName(plain, body)
+ if twin, ok := tree[hashed]; ok && string(twin) == string(body) {
+ out = append(out, m+"/"+hashed)
+ }
+ }
+ sort.Strings(out)
+ return out
+}
diff --git a/internal/gallery/hashed_test.go b/internal/gallery/hashed_test.go
new file mode 100644
index 0000000..bcf3930
--- /dev/null
+++ b/internal/gallery/hashed_test.go
@@ -0,0 +1,38 @@
+package gallery
+
+import (
+ "slices"
+ "testing"
+)
+
+// TestFingerprintedNamesTheHashedTwinsOnly: an asset's hashed name is
+// listed under the mount when the tree holds it beside its plain twin
+// with the same bytes. The plain twin is not listed, since its bytes
+// change under the same name, and neither is a name that merely looks
+// hashed: one whose hex is not its own bytes' hash, or one with no
+// plain twin, could be replaced by other bytes under the same name.
+func TestFingerprintedNamesTheHashedTwinsOnly(t *testing.T) {
+ css, js, pic := []byte("body{}\n"), []byte("0\n"), []byte("GIF89a")
+ tree := map[string][]byte{
+ "assets/css/fiesta.css": css,
+ "assets/css/" + hashedName("fiesta.css", css): css,
+ "assets/gallery.js": js,
+ "assets/" + hashedName("gallery.js", js): js,
+ "assets/images/a.gif": pic,
+ "assets/images/" + hashedName("a.gif", pic): pic,
+ "assets/lone.0123456789abcdef.css": css,
+ "assets/wrong.css": css,
+ "assets/wrong.0123456789abcdef.css": css,
+ "day/en/" + hashedName("page.html", js): js,
+ "day/en/page.html": js,
+ }
+ want := []string{
+ "/x/assets/" + hashedName("gallery.js", js),
+ "/x/assets/css/" + hashedName("fiesta.css", css),
+ "/x/assets/images/" + hashedName("a.gif", pic),
+ }
+ slices.Sort(want)
+ if got := Fingerprinted("/x/", tree); !slices.Equal(got, want) {
+ t.Errorf("Fingerprinted is\n%q\nwant\n%q", got, want)
+ }
+}
diff --git a/internal/site/export_test.go b/internal/site/export_test.go
new file mode 100644
index 0000000..7eeec44
--- /dev/null
+++ b/internal/site/export_test.go
@@ -0,0 +1,8 @@
+package site
+
+// HeadersWith is a _headers file of one path setting one header, put
+// through the check Build's own rules pass, so a test can hand it a
+// header Build never writes.
+func HeadersWith(path, name, value string) ([]byte, error) {
+ return headersFile([]headerGroup{{paths: []string{path}, headers: []header{{name, value}}}})
+}
diff --git a/internal/site/site.go b/internal/site/site.go
index 97d2689..84dc587 100644
--- a/internal/site/site.go
+++ b/internal/site/site.go
@@ -1,6 +1,7 @@
// Package site assembles fiesta.rastrillo.org: the committed releases
// under their exact versions, a rewrite per major onto its newest
-// release, and the pages at the root.
+// release, the response headers that let browsers keep what never
+// changes, and the pages at the root.
//
// Build is a pure function of the two trees it is handed, so the same
// releases and pages are the same site on anyone's machine. cmd/fiesta
@@ -14,6 +15,7 @@ import (
"fmt"
"io/fs"
"regexp"
+ "slices"
"sort"
"strconv"
"strings"
@@ -97,6 +99,19 @@ const (
redirectsMaxLine = 2048
)
+// Rules are what the pages ask of the site's rules files, as
+// internal/www works them out from the pages it assembled.
+type Rules struct {
+ // Redirects are _redirects lines, written one a line as given,
+ // after the release rewrites, so a page's rule never decides a /M/
+ // URL first.
+ Redirects []string
+ // Immutable are pages, by their address on the site, whose name
+ // carries a hash of their bytes, so the bytes at that address can
+ // never change. _headers lets browsers keep each for a year.
+ Immutable []string
+}
+
// less orders versions numerically, so 1.10.0 is newer than 1.9.3.
func (v version) less(w version) bool {
if v.major != w.major {
@@ -116,8 +131,11 @@ func (v version) less(w version) bool {
// against the release's SHA256SUMS first (Release);
// - _redirects, one rule per major, newest major first, sending
// /M/* to that major's newest release: `/M/* /M.y.z/:splat 200`;
-// then own, the pages' rules, one a line as given, so a page's
-// rule never decides a /M/ URL first;
+// then rules.Redirects, the pages' rules, one a line as given, so
+// a page's rule never decides a /M/ URL first;
+// - _headers, a five-minute cache on each major's /M/*, a year's
+// immutable one on each release's /M.y.z/*, both open to other
+// origins, then a year's on each of rules.Immutable (headersFor);
// - every page, at the path it has under pages.
//
// Anything else is an error, and the first error is the whole answer:
@@ -127,10 +145,11 @@ func (v version) less(w version) bool {
// at the request path before any rewrite that is not forced, so a page
// under an all-digit first segment would answer for /M/*, released or
// not, and a page under a version-shaped segment would answer for that
-// release. Both are refused, and so is a page named _redirects, which
-// is the rules file. So is a rule that is not one line of it, or
-// rules that take it past the edge's limits.
-func Build(releases fs.FS, pages fs.FS, own ...string) (map[string][]byte, error) {
+// release. Both are refused, and so is a page named _redirects or
+// _headers, which are the rules files. So is a redirect that is not
+// one line of its file, an immutable address that is not a page's,
+// and rules that take either file past the edge's limits.
+func Build(releases fs.FS, pages fs.FS, rules Rules) (map[string][]byte, error) {
out := map[string][]byte{}
entries, err := fs.ReadDir(releases, ".")
if err != nil {
@@ -171,14 +190,14 @@ func Build(releases fs.FS, pages fs.FS, own ...string) (map[string][]byte, error
if len(majors) > redirectsMaxRules {
return nil, fmt.Errorf("%s would hold %d rules, one per major, and the edge accepts %d", redirectsName, len(majors), redirectsMaxRules)
}
- if n := len(majors) + len(own); n > redirectsMaxRules {
- return nil, fmt.Errorf("%s would hold %d rules, %d for the releases and %d for the pages, and the edge accepts %d", redirectsName, n, len(majors), len(own), redirectsMaxRules)
+ if n := len(majors) + len(rules.Redirects); n > redirectsMaxRules {
+ return nil, fmt.Errorf("%s would hold %d rules, %d for the releases and %d for the pages, and the edge accepts %d", redirectsName, n, len(majors), len(rules.Redirects), redirectsMaxRules)
}
- var rules bytes.Buffer
+ var redirects bytes.Buffer
for _, m := range majors {
- fmt.Fprintf(&rules, "/%d/* /%s/:splat 200\n", m, newest[m].name)
+ fmt.Fprintf(&redirects, "/%d/* /%s/:splat 200\n", m, newest[m].name)
}
- for _, r := range own {
+ for _, r := range rules.Redirects {
switch {
case strings.ContainsAny(r, "\n\r"):
return nil, fmt.Errorf("the %s rule %q is not one line", redirectsName, r)
@@ -187,12 +206,12 @@ func Build(releases fs.FS, pages fs.FS, own ...string) (map[string][]byte, error
case len(r) > redirectsMaxLine:
return nil, fmt.Errorf("the %s rule %.40q... is %d bytes, and the edge accepts %d a line", redirectsName, r, len(r), redirectsMaxLine)
}
- rules.WriteString(r + "\n")
+ redirects.WriteString(r + "\n")
}
- if rules.Len() > redirectsMaxBytes {
- return nil, fmt.Errorf("%s would be %d bytes, and the edge accepts %d", redirectsName, rules.Len(), redirectsMaxBytes)
+ if redirects.Len() > redirectsMaxBytes {
+ return nil, fmt.Errorf("%s would be %d bytes, and the edge accepts %d", redirectsName, redirects.Len(), redirectsMaxBytes)
}
- out[redirectsName] = rules.Bytes()
+ out[redirectsName] = redirects.Bytes()
err = fs.WalkDir(pages, ".", func(p string, d fs.DirEntry, err error) error {
if err != nil {
@@ -220,8 +239,8 @@ func Build(releases fs.FS, pages fs.FS, own ...string) (map[string][]byte, error
return fmt.Errorf("page %s would shadow /%s/, the rewrite onto the newest %s.x release, so no page may start with digits alone", p, first, first)
case strictVersion.MatchString(first):
return fmt.Errorf("page %s would shadow the release directory %s/, so no page may start with a version", p, first)
- case first == redirectsName:
- return fmt.Errorf("page %s is the rules file, which the site writes itself", p)
+ case first == redirectsName || first == headersName:
+ return fmt.Errorf("page %s is a rules file, which the site writes itself", p)
}
if !d.Type().IsRegular() {
return fmt.Errorf("page %s is not a regular file", p)
@@ -236,9 +255,87 @@ func Build(releases fs.FS, pages fs.FS, own ...string) (map[string][]byte, error
if err != nil {
return nil, err
}
+
+ headers, err := headersFor(versions, majors, rules.Immutable, out)
+ if err != nil {
+ return nil, err
+ }
+ out[headersName] = headers
return out, nil
}
+// headersFor is the site's _headers: a short cache on each major's
+// /M/, a year on each release's /M.y.z/, both open to other origins,
+// and a year on each immutable page, in that order. Every file it
+// does not name keeps the edge's no-cache, the pages included, so a
+// deploy shows on the next load.
+//
+// Each immutable address must be a page of the site, not a release's
+// file, written so the edge reads it back as that page's path: rooted,
+// and free of what a path line reads as a pattern (":name", "*", a
+// percent escape), as a query or fragment, or as the line's end
+// (white space). Otherwise the rule would name nothing, or something
+// else, and the page would quietly keep no-cache.
+func headersFor(versions []version, majors []int, immutable []string, out map[string][]byte) ([]byte, error) {
+ ordered := slices.Clone(versions)
+ slices.SortFunc(ordered, func(a, b version) int {
+ switch {
+ case a.less(b):
+ return -1
+ case b.less(a):
+ return 1
+ }
+ return 0
+ })
+ ascending := slices.Clone(majors)
+ slices.Sort(ascending)
+
+ // Majors, then releases, then pages: none of their paths overlap
+ // today, but where one ever does, the later and more specific rule
+ // is the one that wins.
+ majorRules := headerGroup{
+ kind: "majors",
+ why: []string{
+ "/M/ moves to each new M.x release, so browsers keep it five minutes.",
+ "The cross-origin headers let pages on other sites fetch these files,",
+ "load them as modules and pin them with integrity hashes.",
+ },
+ headers: append([]header{{"Cache-Control", cacheBriefly}}, crossOrigin...),
+ }
+ for _, m := range ascending {
+ majorRules.paths = append(majorRules.paths, fmt.Sprintf("/%d/*", m))
+ }
+ releaseRules := headerGroup{
+ kind: "releases",
+ why: []string{"A published release never changes, so browsers keep it for a year."},
+ headers: append([]header{{"Cache-Control", cacheForever}}, crossOrigin...),
+ }
+ for _, v := range ordered {
+ releaseRules.paths = append(releaseRules.paths, "/"+v.name+"/*")
+ }
+ pageRules := headerGroup{
+ kind: "immutable pages",
+ why: []string{"Each of these names carries a hash of its bytes, so it never changes."},
+ headers: []header{{"Cache-Control", cacheForever}},
+ }
+ for _, p := range immutable {
+ name, rooted := strings.CutPrefix(p, "/")
+ first, _, _ := strings.Cut(name, "/")
+ switch {
+ case !rooted:
+ return nil, fmt.Errorf("the immutable address %q is not rooted; write it as the site serves it, from /", p)
+ case strings.ContainsFunc(p, func(r rune) bool { return r <= ' ' || strings.ContainsRune(":*%?#", r) }):
+ return nil, fmt.Errorf("the immutable address %q holds a character a %s path line reads as more than a path", p, headersName)
+ }
+ if _, ok := out[name]; !ok || strictVersion.MatchString(first) {
+ return nil, fmt.Errorf("the immutable address %q is not a page of the site", p)
+ }
+ pageRules.paths = append(pageRules.paths, p)
+ }
+ slices.Sort(pageRules.paths)
+ return headersFile([]headerGroup{majorRules, releaseRules, pageRules})
+}
+
// Release reads one release directory and checks it against its
// SHA256SUMS: every file is listed and its bytes match, and every
// listed file is present. It returns the files keyed by path, the sums
@@ -321,3 +418,151 @@ func Release(dir fs.FS) (map[string][]byte, error) {
}
return files, nil
}
+
+// headersName is the file of response headers by path at the root of
+// a static site, in the format Netlify and Cloudflare Pages use and
+// CARLOS's edge serves from v0.31.1. Build writes it beside
+// redirectsName.
+const headersName = "_headers"
+
+// The edge's limits on one _headers file, from CARLOS's
+// internal/edge/headers.go: 100 path lines and 2,048 characters a
+// line. Its third limit, 256 KiB, cannot be reached under the other
+// two: 90 paths of 2,048 characters, with the few fixed header lines
+// under each, come to under 200 KiB.
+//
+// Build stops at headersStopPaths, ten short of the edge, so the file
+// is reshaped while there is room, not on the day a release must ship.
+const (
+ headersMaxPaths = 100
+ headersStopPaths = 90
+ headersMaxLine = 2048
+)
+
+// The Cache-Control values Build writes. immutable stops a browser
+// revalidating even on a reload, so it goes only where the bytes at
+// the address can never change: a release, and a name that carries
+// its bytes' hash.
+const (
+ cacheForever = "public, max-age=31536000, immutable"
+ cacheBriefly = "public, max-age=300"
+ headersIntro = "# Response headers for CARLOS's edge, written by fiesta site.\n" +
+ "# A rule matches the address the browser asked for, before _redirects\n" +
+ "# rewrites it. Where two rules set a header, the later one wins.\n" +
+ "# A file no rule names keeps the edge's Cache-Control: no-cache.\n"
+)
+
+// crossOrigin opens a release's files to pages on other sites. The
+// first lets them fetch the strings, load a module script and pin a
+// file by its integrity hash; the second lets a page that isolates
+// itself (Cross-Origin-Embedder-Policy) load the files at all. The
+// gallery's own assets, which only the gallery's pages load, go
+// without.
+var crossOrigin = []header{
+ {"Access-Control-Allow-Origin", "*"},
+ {"Cross-Origin-Resource-Policy", "cross-origin"},
+}
+
+// header is one header line of a _headers rule.
+type header struct{ name, value string }
+
+// headerGroup is a run of _headers rules that set the same headers,
+// under a comment that says why.
+type headerGroup struct {
+ kind string // what the paths are, for a refusal's count
+ why []string // the comment's lines
+ paths []string
+ headers []header
+}
+
+// headerChecks are the headers CARLOS's edge lets a site set, by the
+// name Build writes, each with the values the edge accepts. carlos
+// ship refuses a file with any other, so Build refuses it first: the
+// headers are constants here, and one edited into something the edge
+// refuses fails the build, not the ship. Cache-Control is held a
+// little tighter than the edge holds it: no quoted values, which
+// Build never writes.
+var headerChecks = map[string]func(string) bool{
+ "Cache-Control": func(v string) bool { return tokenList(v, true) },
+ "Access-Control-Allow-Origin": func(v string) bool { return v == "*" },
+ "Access-Control-Expose-Headers": func(v string) bool { return tokenList(v, false) },
+ "Cross-Origin-Resource-Policy": func(v string) bool {
+ return v == "same-origin" || v == "same-site" || v == "cross-origin"
+ },
+ "Timing-Allow-Origin": func(v string) bool { return v == "*" },
+}
+
+// tokenList reports whether v is a comma-separated list of HTTP tokens
+// (RFC 9110), each optionally token=token when assign is set.
+func tokenList(v string, assign bool) bool {
+ for _, item := range strings.Split(v, ",") {
+ item = strings.Trim(item, " ")
+ if name, val, ok := strings.Cut(item, "="); ok && assign {
+ if !isToken(name) || !isToken(val) {
+ return false
+ }
+ } else if !isToken(item) {
+ return false
+ }
+ }
+ return true
+}
+
+// isToken reports whether s is one or more of the characters an HTTP
+// header or directive name may hold.
+func isToken(s string) bool {
+ if s == "" {
+ return false
+ }
+ for i := 0; i < len(s); i++ {
+ c := s[i]
+ if !(c >= '0' && c <= '9' || c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || strings.IndexByte("!#$%&'*+-.^_`|~", c) >= 0) {
+ return false
+ }
+ }
+ return true
+}
+
+// headersFile writes groups as a _headers file, in order, or refuses
+// what the edge would refuse: more than headersStopPaths paths, a line
+// over headersMaxLine, or a header outside headerChecks.
+func headersFile(groups []headerGroup) ([]byte, error) {
+ var counts []string
+ n := 0
+ for _, g := range groups {
+ if len(g.paths) > 0 {
+ n += len(g.paths)
+ counts = append(counts, fmt.Sprintf("%d %s", len(g.paths), g.kind))
+ }
+ }
+ if n > headersStopPaths {
+ return nil, fmt.Errorf("%s would name %d paths (%s), and the build stops at %d, short of the edge's limit of %d; "+
+ "give the gallery's hashed assets a directory of their own so one rule covers them, or ask CARLOS for a higher limit",
+ headersName, n, strings.Join(counts, ", "), headersStopPaths, headersMaxPaths)
+ }
+ var b bytes.Buffer
+ b.WriteString(headersIntro)
+ for _, g := range groups {
+ if len(g.paths) == 0 {
+ continue
+ }
+ b.WriteString("\n")
+ for _, line := range g.why {
+ b.WriteString("# " + line + "\n")
+ }
+ for _, p := range g.paths {
+ if len(p) > headersMaxLine {
+ return nil, fmt.Errorf("the %s path %.40q... is %d characters, and the edge accepts %d a line", headersName, p, len(p), headersMaxLine)
+ }
+ b.WriteString(p + "\n")
+ for _, h := range g.headers {
+ if check := headerChecks[h.name]; check == nil || !check(h.value) {
+ return nil, fmt.Errorf("%s would set %s: %q on %s, and the edge refuses it; "+
+ "set only the headers in headerChecks, with the values it allows", headersName, h.name, h.value, p)
+ }
+ b.WriteString(" " + h.name + ": " + h.value + "\n")
+ }
+ }
+ }
+ return b.Bytes(), nil
+}
diff --git a/internal/site/site_test.go b/internal/site/site_test.go
index f275207..c149faf 100644
--- a/internal/site/site_test.go
+++ b/internal/site/site_test.go
@@ -81,7 +81,7 @@ func TestOneServesTheNewestOneX(t *testing.T) {
"index.html": "<h1>Fiesta</h1>\n",
"themes/index.html": "<h1>Themes</h1>\n",
})
- out, err := site.Build(releases, pages)
+ out, err := site.Build(releases, pages, site.Rules{})
if err != nil {
t.Fatalf("Build: %v", err)
}
@@ -103,20 +103,20 @@ func TestOneServesTheNewestOneX(t *testing.T) {
t.Errorf("page %s differs from the input", name)
}
}
- if want := len(releases) + len(pages) + 1; len(out) != want {
+ if want := len(releases) + len(pages) + 2; len(out) != want {
names := make([]string, 0, len(out))
for name := range out {
names = append(names, name)
}
sort.Strings(names)
- t.Errorf("the site holds %d files, want %d (the releases, the pages and _redirects):\n%s", len(out), want, strings.Join(names, "\n"))
+ t.Errorf("the site holds %d files, want %d (the releases, the pages, _redirects and _headers):\n%s", len(out), want, strings.Join(names, "\n"))
}
}
// TestNewestIsNumericNotLexical: 1.10.0 is newer than 1.9.3, which a
// string sort gets wrong.
func TestNewestIsNumericNotLexical(t *testing.T) {
- out, err := site.Build(releasesOf("1.2.0", "1.10.0", "1.9.3"), pagesOf(nil))
+ out, err := site.Build(releasesOf("1.2.0", "1.10.0", "1.9.3"), pagesOf(nil), site.Rules{})
if err != nil {
t.Fatalf("Build: %v", err)
}
@@ -131,7 +131,7 @@ func TestNewestIsNumericNotLexical(t *testing.T) {
// file is compared, so the format is pinned: two spaces between
// fields, one line per major, nothing else.
func TestTwoDoesNotMoveOne(t *testing.T) {
- out, err := site.Build(releasesOf("1.0.0", "1.1.0", "2.0.0"), pagesOf(nil))
+ out, err := site.Build(releasesOf("1.0.0", "1.1.0", "2.0.0"), pagesOf(nil), site.Rules{})
if err != nil {
t.Fatalf("Build: %v", err)
}
@@ -171,7 +171,7 @@ func TestAChangedReleaseIsRefused(t *testing.T) {
t.Run(c.name, func(t *testing.T) {
releases := releasesOf("1.0.0", "1.1.0")
c.change(releases)
- out, err := site.Build(releases, pagesOf(nil))
+ out, err := site.Build(releases, pagesOf(nil), site.Rules{})
if err == nil {
t.Fatalf("Build published a release with %s", c.name)
}
@@ -195,7 +195,7 @@ func TestAPrereleaseDirectoryIsRefused(t *testing.T) {
for file, data := range release("1.1.0") {
releases[name+"/"+file] = &fstest.MapFile{Data: data}
}
- _, err := site.Build(releases, pagesOf(nil))
+ _, err := site.Build(releases, pagesOf(nil), site.Rules{})
if err == nil {
t.Fatalf("Build accepted releases/%s/", name)
}
@@ -207,7 +207,7 @@ func TestAPrereleaseDirectoryIsRefused(t *testing.T) {
t.Run("a file", func(t *testing.T) {
releases := releasesOf("1.0.0")
releases["README.md"] = &fstest.MapFile{Data: []byte("# releases\n")}
- _, err := site.Build(releases, pagesOf(nil))
+ _, err := site.Build(releases, pagesOf(nil), site.Rules{})
if err == nil || !strings.Contains(err.Error(), "README.md") {
t.Errorf("Build did not refuse a loose file at the top of releases/ by name: %v", err)
}
@@ -222,7 +222,7 @@ func TestPagesCannotShadowAMajor(t *testing.T) {
for _, page := range []string{"1/fiesta.css", "7/x.html", "3"} {
t.Run(page, func(t *testing.T) {
pages := pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n", page: "shadow"})
- _, err := site.Build(releasesOf("1.0.0"), pages)
+ _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{})
if err == nil {
t.Fatalf("Build accepted the page %s", page)
}
@@ -234,13 +234,13 @@ func TestPagesCannotShadowAMajor(t *testing.T) {
}
// TestPagesCannotShadowAVersion: a page under a version directory,
-// released or merely shaped like one, and a page named _redirects are
-// refused by name.
+// released or merely shaped like one, and a page named _redirects or
+// _headers are refused by name.
func TestPagesCannotShadowAVersion(t *testing.T) {
- for _, page := range []string{"1.0.0/index.html", "9.9.9/fiesta.css", "_redirects"} {
+ for _, page := range []string{"1.0.0/index.html", "9.9.9/fiesta.css", "_redirects", "_headers"} {
t.Run(page, func(t *testing.T) {
pages := pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n", page: "shadow"})
- _, err := site.Build(releasesOf("1.0.0"), pages)
+ _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{})
if err == nil {
t.Fatalf("Build accepted the page %s", page)
}
@@ -263,7 +263,7 @@ func TestWhatBuildWritesPassesTheCheck(t *testing.T) {
for name, data := range dist {
releases[fiesta.Version+"/"+name] = &fstest.MapFile{Data: data}
}
- out, err := site.Build(releases, pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n"}))
+ out, err := site.Build(releases, pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n"}), site.Rules{})
if err != nil {
t.Fatalf("Build refused the real dist: %v", err)
}
@@ -293,7 +293,7 @@ func TestTheStampIsSkippedAndNeverPublished(t *testing.T) {
releases := releasesOf("1.0.0")
stamp := "1.0.0/" + site.StampName
releases[stamp] = &fstest.MapFile{Data: []byte("This directory is written by fiesta release, which EMPTIES it on every\nrun.\n")}
- out, err := site.Build(releases, pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n"}))
+ out, err := site.Build(releases, pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n"}), site.Rules{})
if err != nil {
t.Fatalf("Build refused a release carrying its stamp: %v", err)
}
@@ -302,14 +302,14 @@ func TestTheStampIsSkippedAndNeverPublished(t *testing.T) {
t.Errorf("the stamp was published as %s", name)
}
}
- if want := len(releases) - 1 + 1 + 1; len(out) != want {
- t.Errorf("the site holds %d files, want %d (the release without its stamp, the page and _redirects)", len(out), want)
+ if want := len(releases) - 1 + 1 + 2; len(out) != want {
+ t.Errorf("the site holds %d files, want %d (the release without its stamp, the page, _redirects and _headers)", len(out), want)
}
for _, stray := range []string{"1.0.0/i18n/" + site.StampName, "1.0.0/.DS_Store"} {
t.Run(stray, func(t *testing.T) {
releases := releasesOf("1.0.0")
releases[stray] = &fstest.MapFile{Data: []byte("x")}
- _, err := site.Build(releases, pagesOf(nil))
+ _, err := site.Build(releases, pagesOf(nil), site.Rules{})
if err == nil {
t.Fatalf("Build published a release holding %s", stray)
}
@@ -329,7 +329,7 @@ func TestTheStampIsSkippedAndNeverPublished(t *testing.T) {
for name, data := range files {
releases["1.0.0/"+name] = &fstest.MapFile{Data: data}
}
- out, err := site.Build(releases, pagesOf(nil))
+ out, err := site.Build(releases, pagesOf(nil), site.Rules{})
if err == nil {
_, published := out["1.0.0/.secret"]
t.Fatalf("Build accepted a release whose SHA256SUMS lists .secret (published: %v)", published)
@@ -354,7 +354,7 @@ func TestHiddenPagesAreRefused(t *testing.T) {
} {
t.Run(page, func(t *testing.T) {
pages := pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n", page: "hidden"})
- _, err := site.Build(releasesOf("1.0.0"), pages)
+ _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{})
if err == nil {
t.Fatalf("Build published the hidden page %s", page)
}
@@ -366,7 +366,7 @@ func TestHiddenPagesAreRefused(t *testing.T) {
t.Run("an empty hidden directory", func(t *testing.T) {
pages := pagesOf(map[string]string{"index.html": "<h1>Fiesta</h1>\n"})
pages[".cache"] = &fstest.MapFile{Mode: fs.ModeDir | 0o755}
- _, err := site.Build(releasesOf("1.0.0"), pages)
+ _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{})
if err == nil || !strings.Contains(err.Error(), ".cache") {
t.Errorf("Build did not refuse the hidden directory .cache by name: %v", err)
}
@@ -416,7 +416,7 @@ func TestALinkedSumsFileIsRefused(t *testing.T) {
// 32-bit build as well.
func TestAVersionComponentMustFit(t *testing.T) {
most := strconv.Itoa(math.MaxInt)
- out, err := site.Build(releasesOf("1.2.0", "1."+most+".0"), pagesOf(nil))
+ out, err := site.Build(releasesOf("1.2.0", "1."+most+".0"), pagesOf(nil), site.Rules{})
if err != nil {
t.Fatalf("Build with a minor of %s: %v", most, err)
}
@@ -425,7 +425,7 @@ func TestAVersionComponentMustFit(t *testing.T) {
}
over := new(big.Int).Add(big.NewInt(math.MaxInt), big.NewInt(1)).String()
for _, name := range []string{"1." + over + ".0", over + ".0.0", "1.0." + over} {
- _, err := site.Build(releasesOf("1.2.0", name), pagesOf(nil))
+ _, err := site.Build(releasesOf("1.2.0", name), pagesOf(nil), site.Rules{})
if err == nil {
t.Errorf("Build accepted releases/%s, whose component does not fit an int", name)
continue
@@ -436,15 +436,19 @@ func TestAVersionComponentMustFit(t *testing.T) {
}
// Two overflowing minors: refused, not read as one number.
over2 := new(big.Int).Add(big.NewInt(math.MaxInt), big.NewInt(2)).String()
- if _, err := site.Build(releasesOf("1."+over+".0", "1."+over2+".0"), pagesOf(nil)); err == nil {
+ if _, err := site.Build(releasesOf("1."+over+".0", "1."+over2+".0"), pagesOf(nil), site.Rules{}); err == nil {
t.Error("Build accepted two releases whose minors both overflow, which Atoi reads as one number")
}
}
// TestTheRedirectsStayUnderTheEdgesLimits: one rule per major, and the
// edge accepts 2,000 rules in a _redirects file (CARLOS's
-// redirectsMaxRules). 2,000 majors build; 2,001 are refused here, by
-// the figure, rather than assembling a site the packer rejects.
+// redirectsMaxRules). A major's rule and 1,999 of the pages' build;
+// 2,001 majors are refused here, by the figure, rather than assembling
+// a site the packer rejects. (_headers, with a path per major and per
+// release, stops a site at 45 majors long before, which
+// TestTheHeadersStopShortOfTheEdgesLimit holds; this is the
+// _redirects check, which comes first.)
func TestTheRedirectsStayUnderTheEdgesLimits(t *testing.T) {
majors := func(n int) []string {
names := make([]string, 0, n)
@@ -453,14 +457,18 @@ func TestTheRedirectsStayUnderTheEdgesLimits(t *testing.T) {
}
return names
}
- out, err := site.Build(releasesOf(majors(2000)...), pagesOf(nil))
+ own := make([]string, 1999)
+ for i := range own {
+ own[i] = "/a" + strconv.Itoa(i) + " /b 301"
+ }
+ out, err := site.Build(releasesOf("1.0.0"), pagesOf(nil), site.Rules{Redirects: own})
if err != nil {
- t.Fatalf("Build with 2000 majors: %v", err)
+ t.Fatalf("Build with 2000 rules: %v", err)
}
if n := strings.Count(string(out["_redirects"]), "\n"); n != 2000 {
t.Errorf("_redirects holds %d rules, want 2000", n)
}
- _, err = site.Build(releasesOf(majors(2001)...), pagesOf(nil))
+ _, err = site.Build(releasesOf(majors(2001)...), pagesOf(nil), site.Rules{})
if err == nil {
t.Fatal("Build accepted 2001 majors: 2001 rules, past the edge's 2000")
}
@@ -473,9 +481,10 @@ func TestTheRedirectsStayUnderTheEdgesLimits(t *testing.T) {
// handed go into _redirects as they are, one a line, in the order
// given, after every release rewrite, so /M/ is decided first.
func TestThePagesOwnRulesFollowTheReleaseRewrites(t *testing.T) {
- out, err := site.Build(releasesOf("1.0.0", "2.0.0"), pagesOf(nil),
+ out, err := site.Build(releasesOf("1.0.0", "2.0.0"), pagesOf(nil), site.Rules{Redirects: []string{
"/old/:page /new/:page 301",
- "/gone.html /here.html 301")
+ "/gone.html /here.html 301",
+ }})
if err != nil {
t.Fatalf("Build: %v", err)
}
@@ -514,7 +523,7 @@ func TestARuleThatIsNotOneLineIsRefused(t *testing.T) {
{"2001 rules with the release's", many, "2000"},
{"over 256 KiB", wide, "bytes"},
} {
- _, err := site.Build(releasesOf("1.0.0"), pagesOf(nil), c.rules...)
+ _, err := site.Build(releasesOf("1.0.0"), pagesOf(nil), site.Rules{Redirects: c.rules})
if err == nil {
t.Errorf("%s: Build accepted it", c.name)
} else if !strings.Contains(err.Error(), c.want) {
@@ -522,3 +531,208 @@ func TestARuleThatIsNotOneLineIsRefused(t *testing.T) {
}
}
}
+
+// The rules every _headers group carries, as the edge reads them.
+const (
+ shortLived = " Cache-Control: public, max-age=300\n"
+ yearLong = " Cache-Control: public, max-age=31536000, immutable\n"
+ cors = " Access-Control-Allow-Origin: *\n" +
+ " Cross-Origin-Resource-Policy: cross-origin\n"
+)
+
+// TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes: _headers
+// gives each major a five-minute cache, since /M/ moves to every new
+// M.x, each release a year's immutable one, since a published release
+// never changes, both with the cross-origin headers, and each
+// immutable page a year's cache alone. The whole file is compared, so
+// the order is pinned too: majors, then releases, then pages, and
+// nothing for the rest of the site, which keeps the edge's no-cache.
+func TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes(t *testing.T) {
+ pages := pagesOf(map[string]string{
+ "index.html": "<h1>Fiesta</h1>\n",
+ "a/x.0123456789abcdef.css": "body{}\n",
+ "a/y.fedcba9876543210.js": "0\n",
+ "a/x.css": "body{}\n",
+ "static/site.css": "main{}\n",
+ "design-system/index.html": "<h1>Gallery</h1>\n",
+ "design-system/en/one.html": "<h1>One</h1>\n",
+ })
+ out, err := site.Build(releasesOf("1.0.0", "1.10.0", "1.9.3", "2.0.0"), pages, site.Rules{
+ Immutable: []string{"/a/y.fedcba9876543210.js", "/a/x.0123456789abcdef.css"},
+ })
+ if err != nil {
+ t.Fatalf("Build: %v", err)
+ }
+ const want = "# Response headers for CARLOS's edge, written by fiesta site.\n" +
+ "# A rule matches the address the browser asked for, before _redirects\n" +
+ "# rewrites it. Where two rules set a header, the later one wins.\n" +
+ "# A file no rule names keeps the edge's Cache-Control: no-cache.\n" +
+ "\n" +
+ "# /M/ moves to each new M.x release, so browsers keep it five minutes.\n" +
+ "# The cross-origin headers let pages on other sites fetch these files,\n" +
+ "# load them as modules and pin them with integrity hashes.\n" +
+ "/1/*\n" + shortLived + cors +
+ "/2/*\n" + shortLived + cors +
+ "\n" +
+ "# A published release never changes, so browsers keep it for a year.\n" +
+ "/1.0.0/*\n" + yearLong + cors +
+ "/1.9.3/*\n" + yearLong + cors +
+ "/1.10.0/*\n" + yearLong + cors +
+ "/2.0.0/*\n" + yearLong + cors +
+ "\n" +
+ "# Each of these names carries a hash of its bytes, so it never changes.\n" +
+ "/a/x.0123456789abcdef.css\n" + yearLong +
+ "/a/y.fedcba9876543210.js\n" + yearLong
+ if got := string(out["_headers"]); got != want {
+ t.Errorf("_headers is\n%s\nwant\n%s", got, want)
+ }
+}
+
+// TestANewReleaseAddsItsOwnHeadersRule: the rules come from the
+// releases built, so cutting 1.1.0 adds /1.1.0/* and nothing else,
+// and /1/* stays one rule.
+func TestANewReleaseAddsItsOwnHeadersRule(t *testing.T) {
+ paths := func(versions ...string) []string {
+ t.Helper()
+ out, err := site.Build(releasesOf(versions...), pagesOf(nil), site.Rules{})
+ if err != nil {
+ t.Fatalf("Build: %v", err)
+ }
+ var ps []string
+ for _, line := range strings.Split(string(out["_headers"]), "\n") {
+ if strings.HasPrefix(line, "/") {
+ ps = append(ps, line)
+ }
+ }
+ return ps
+ }
+ before, after := paths("1.0.0"), paths("1.0.0", "1.1.0")
+ if got, want := strings.Join(before, " "), "/1/* /1.0.0/*"; got != want {
+ t.Errorf("with 1.0.0, _headers names %s; want %s", got, want)
+ }
+ if got, want := strings.Join(after, " "), "/1/* /1.0.0/* /1.1.0/*"; got != want {
+ t.Errorf("with 1.0.0 and 1.1.0, _headers names %s; want %s", got, want)
+ }
+}
+
+// TestAnImmutableAddressMustBeAPage: an immutable address names one
+// page the site serves, spelled so the edge reads it as that page. A
+// path that is no page, a release file, which has its release's rule,
+// a relative one, and one holding a character the edge reads as a
+// pattern (":", "*", "%"), a query or white space are refused by name.
+func TestAnImmutableAddressMustBeAPage(t *testing.T) {
+ pages := pagesOf(map[string]string{
+ "index.html": "<h1>Fiesta</h1>\n",
+ "a/x.1.css": "body{}\n",
+ "a/:x.css": "body{}\n",
+ "a/*.css": "body{}\n",
+ "a/%41.css": "body{}\n",
+ "a/x y.css": "body{}\n",
+ "a/x?.css": "body{}\n",
+ "a/x#.css": "body{}\n",
+ "a/tab\t.css": "body{}\n",
+ })
+ for _, p := range []string{
+ "/a/missing.css", "/1.0.0/fiesta.css", "a/x.1.css", "/a/:x.css",
+ "/a/*.css", "/a/%41.css", "/a/x y.css", "/a/x?.css", "/a/x#.css", "/a/tab\t.css",
+ } {
+ _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{Immutable: []string{p}})
+ if err == nil {
+ t.Errorf("Build accepted the immutable address %q", p)
+ } else if !strings.Contains(err.Error(), strconv.Quote(p)) {
+ t.Errorf("the refusal of %q does not name it: %v", p, err)
+ }
+ }
+ if _, err := site.Build(releasesOf("1.0.0"), pages, site.Rules{Immutable: []string{"/a/x.1.css"}}); err != nil {
+ t.Errorf("Build refused a page's address: %v", err)
+ }
+}
+
+// TestTheHeadersStopShortOfTheEdgesLimit: the edge refuses a _headers
+// file of more than 100 paths, or a line over 2,048 characters. Build
+// stops at 90 paths, whether releases or pages take it there, and says
+// what to do; and refuses a page whose path line is too long.
+func TestTheHeadersStopShortOfTheEdgesLimit(t *testing.T) {
+ releases := func(n int) []string {
+ names := make([]string, 0, n)
+ for i := 0; i < n; i++ {
+ names = append(names, "1."+strconv.Itoa(i)+".0")
+ }
+ return names
+ }
+ // One major and 89 releases: 90 paths, the most Build writes.
+ out, err := site.Build(releasesOf(releases(89)...), pagesOf(nil), site.Rules{})
+ if err != nil {
+ t.Fatalf("Build with 90 paths: %v", err)
+ }
+ if n := strings.Count(string(out["_headers"]), "\n/"); n != 90 {
+ t.Errorf("_headers names %d paths, want 90", n)
+ }
+ _, err = site.Build(releasesOf(releases(90)...), pagesOf(nil), site.Rules{})
+ if err == nil {
+ t.Fatal("Build wrote 91 paths into _headers")
+ }
+ for _, want := range []string{"91 paths", "90", "100", "directory of their own"} {
+ if !strings.Contains(err.Error(), want) {
+ t.Errorf("the refusal does not say %q: %v", want, err)
+ }
+ }
+
+ files := map[string]string{"index.html": "<h1>Fiesta</h1>\n"}
+ var immutable []string
+ for i := 0; i < 89; i++ {
+ name := "a/x" + strconv.Itoa(i) + ".css"
+ files[name] = "body{}\n"
+ immutable = append(immutable, "/"+name)
+ }
+ // One major, one release and 89 pages.
+ _, err = site.Build(releasesOf("1.0.0"), pagesOf(files), site.Rules{Immutable: immutable})
+ if err == nil || !strings.Contains(err.Error(), "91 paths") {
+ t.Errorf("Build did not stop at 91 paths, 89 of them pages: %v", err)
+ }
+
+ long := "a/" + strings.Repeat("x", 2048) + ".css"
+ _, err = site.Build(releasesOf("1.0.0"), pagesOf(map[string]string{long: "body{}\n"}),
+ site.Rules{Immutable: []string{"/" + long}})
+ if err == nil || !strings.Contains(err.Error(), "2048") {
+ t.Errorf("Build did not refuse a path line over 2,048 characters: %v", err)
+ }
+}
+
+// TestTheHeadersSetOnlyWhatTheEdgeAllows: CARLOS sends five headers
+// from a _headers file and refuses any other, and refuses an origin
+// other than "*". Build's own rules pass the same check as these, so
+// a header edited into site.go that the edge would refuse fails the
+// build, not carlos ship.
+func TestTheHeadersSetOnlyWhatTheEdgeAllows(t *testing.T) {
+ for _, c := range []struct {
+ name, value string
+ ok bool
+ }{
+ {"Cache-Control", "public, max-age=300", true},
+ {"Cache-Control", "public, max-age=31536000, immutable", true},
+ {"Access-Control-Allow-Origin", "*", true},
+ {"Access-Control-Expose-Headers", "Content-Length, ETag", true},
+ {"Cross-Origin-Resource-Policy", "cross-origin", true},
+ {"Timing-Allow-Origin", "*", true},
+ {"Set-Cookie", "a=b", false},
+ {"Content-Security-Policy", "default-src 'self'", false},
+ {"Strict-Transport-Security", "max-age=63072000", false},
+ {"Access-Control-Allow-Origin", "https://example.com", false},
+ {"Timing-Allow-Origin", "https://example.com", false},
+ {"Cross-Origin-Resource-Policy", "anywhere", false},
+ {"Cache-Control", "public max-age=300", false},
+ {"Cache-Control", "", false},
+ {"Access-Control-Expose-Headers", "Content Length", false},
+ } {
+ _, err := site.HeadersWith("/1/*", c.name, c.value)
+ switch {
+ case c.ok && err != nil:
+ t.Errorf("%s: %s was refused: %v", c.name, c.value, err)
+ case !c.ok && err == nil:
+ t.Errorf("%s: %s was accepted; the edge refuses it", c.name, c.value)
+ case !c.ok && !strings.Contains(err.Error(), c.name):
+ t.Errorf("the refusal of %s: %s does not name the header: %v", c.name, c.value, err)
+ }
+ }
+}
diff --git a/internal/www/headers_test.go b/internal/www/headers_test.go
new file mode 100644
index 0000000..fc57fd3
--- /dev/null
+++ b/internal/www/headers_test.go
@@ -0,0 +1,380 @@
+package www_test
+
+// The _headers grammar and matching of CARLOS's static edge, mirrored
+// from internal/edge/headers.go in carlos/platform (v0.31.1) so the
+// tests can refuse a file `carlos ship` would refuse and read the
+// headers a request gets. When the edge changes its grammar, this
+// changes with it.
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "net/http"
+ "path"
+ "regexp"
+ "slices"
+ "strings"
+ "testing"
+
+ "amadan.net/rastrillo/fiesta"
+)
+
+// The edge's limits on one _headers file.
+const (
+ headersMaxBytes = 256 << 10
+ headersMaxPaths = 100
+ headersMaxLine = 2048
+)
+
+// headerRule is one path line of a _headers file and the headers under
+// it, in the order written.
+type headerRule struct {
+ line int
+ path string
+ rule // the path, read as a _redirects source is
+ set http.Header
+}
+
+// headerChecks are the headers the edge lets a site set, each with the
+// check its value must pass.
+var headerChecks = map[string]func(string) bool{
+ "Cache-Control": cacheDirectives,
+ "Access-Control-Allow-Origin": func(v string) bool { return v == "*" },
+ "Access-Control-Expose-Headers": headerNames,
+ "Cross-Origin-Resource-Policy": func(v string) bool {
+ return v == "same-origin" || v == "same-site" || v == "cross-origin"
+ },
+ "Timing-Allow-Origin": func(v string) bool { return v == "*" },
+}
+
+// parseHeaders parses a _headers file as `carlos ship` does: every
+// line it cannot use is an error, named by number.
+func parseHeaders(data []byte) ([]headerRule, error) {
+ if len(data) > headersMaxBytes {
+ return nil, fmt.Errorf("_headers is %d bytes; the limit is %d", len(data), headersMaxBytes)
+ }
+ var rules []headerRule
+ var errs []error
+ bad := func(n int, err error) { errs = append(errs, fmt.Errorf("line %d: %v", n, err)) }
+ var cur *headerRule
+ seenPath, paths := false, 0
+ finish := func() {
+ if cur != nil && len(cur.set) == 0 {
+ bad(cur.line, fmt.Errorf("path %q has no headers under it", cur.path))
+ } else if cur != nil {
+ rules = append(rules, *cur)
+ }
+ cur = nil
+ }
+ text := strings.TrimPrefix(string(data), "\ufeff")
+ for i, line := range strings.Split(text, "\n") {
+ n := i + 1
+ line = strings.TrimSuffix(line, "\r")
+ trimmed := strings.Trim(line, " \t")
+ if len(line) > headersMaxLine {
+ bad(n, fmt.Errorf("longer than %d characters", headersMaxLine))
+ if trimmed != "" && line[0] != ' ' && line[0] != '\t' && !strings.HasPrefix(trimmed, "#") {
+ finish()
+ seenPath = true
+ paths++
+ }
+ continue
+ }
+ if trimmed == "" || strings.HasPrefix(trimmed, "#") {
+ continue
+ }
+ if line[0] != ' ' && line[0] != '\t' {
+ finish()
+ seenPath = true
+ paths++
+ switch {
+ case strings.ContainsAny(trimmed, " \t"):
+ bad(n, fmt.Errorf("path line %q: want a path and nothing else", trimmed))
+ continue
+ case paths > headersMaxPaths:
+ bad(n, fmt.Errorf("more than %d path lines", headersMaxPaths))
+ continue
+ }
+ // The edge reads a path line with the parser it reads a
+ // _redirects source with, so the source half of a rule
+ // is that parser here.
+ r, err := parseRedirectLine(trimmed + " /")
+ if err != nil {
+ bad(n, err)
+ continue
+ }
+ cur = &headerRule{line: n, path: trimmed, rule: r, set: http.Header{}}
+ continue
+ }
+ if !seenPath {
+ bad(n, errors.New("header line before a path line"))
+ continue
+ }
+ if cur == nil {
+ continue
+ }
+ name, value, ok := strings.Cut(trimmed, ":")
+ name, value = http.CanonicalHeaderKey(strings.Trim(name, " \t")), strings.Trim(value, " \t")
+ check, known := headerChecks[name]
+ switch {
+ case !ok || name == "" || strings.HasPrefix(name, "!"):
+ bad(n, fmt.Errorf("%q: want a header line like \"Name: value\"", trimmed))
+ case !known:
+ bad(n, fmt.Errorf("header %q is not supported", name))
+ case value == "" || !check(value):
+ bad(n, fmt.Errorf("%s: the edge refuses %q", name, value))
+ default:
+ cur.set.Set(name, value)
+ }
+ }
+ finish()
+ return rules, errors.Join(errs...)
+}
+
+// isToken reports whether s is an HTTP token (RFC 9110).
+func isToken(s string) bool {
+ if s == "" {
+ return false
+ }
+ for i := 0; i < len(s); i++ {
+ c := s[i]
+ if !(c >= '0' && c <= '9' || c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || strings.IndexByte("!#$%&'*+-.^_`|~", c) >= 0) {
+ return false
+ }
+ }
+ return true
+}
+
+// cacheDirectives: a comma-separated list of directives, each a token
+// or token=value, the value a token or a quoted string.
+func cacheDirectives(v string) bool {
+ for _, d := range strings.Split(v, ",") {
+ name, val, hasVal := strings.Cut(strings.Trim(d, " \t"), "=")
+ quoted := len(val) >= 2 && val[0] == '"' && val[len(val)-1] == '"' && !strings.ContainsAny(val[1:len(val)-1], `"\`)
+ if !isToken(name) || hasVal && !isToken(val) && !quoted {
+ return false
+ }
+ }
+ return true
+}
+
+// headerNames: a comma-separated list of header names.
+func headerNames(v string) bool {
+ for _, n := range strings.Split(v, ",") {
+ if !isToken(strings.Trim(n, " \t")) {
+ return false
+ }
+ }
+ return true
+}
+
+// lookupHeaders is what the edge adds to a file served for host at the
+// cleaned path p, the path the browser asked for: every matching
+// rule's headers in file order, the last to name a header winning.
+func lookupHeaders(rules []headerRule, host, p string) http.Header {
+ out := http.Header{}
+ segs := segments(p)
+ for i := range rules {
+ if _, ok := rules[i].match(host, segs); !ok {
+ continue
+ }
+ for name := range rules[i].set {
+ out.Set(name, rules[i].set.Get(name))
+ }
+ }
+ return out
+}
+
+// The headers fiesta site sets, as the edge reads them.
+const (
+ yearLong = "public, max-age=31536000, immutable"
+ fiveMinute = "public, max-age=300"
+)
+
+// crossOrigin are the headers that let a page on another site fetch a
+// release's files, load them as modules and pin them by hash.
+var crossOrigin = http.Header{
+ "Access-Control-Allow-Origin": {"*"},
+ "Cross-Origin-Resource-Policy": {"cross-origin"},
+}
+
+// withCache is h with a Cache-Control of cc added, h left alone.
+func withCache(h http.Header, cc string) http.Header {
+ out := h.Clone()
+ if out == nil {
+ out = http.Header{}
+ }
+ out.Set("Cache-Control", cc)
+ return out
+}
+
+// versionDir is a release directory's name, X.Y.Z, as site.Build reads it.
+var versionDir = regexp.MustCompile(`^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$`)
+
+// hashedAsset is a design-system asset's hashed name: its stem, sixteen
+// hex digits and its extension.
+var hashedAsset = regexp.MustCompile(`^design-system/assets/.*\.([0-9a-f]{16})\.[^./]+$`)
+
+// fingerprinted reports whether the file at name is one of the design
+// system's hashed assets: its name carries the first sixteen hex
+// digits of its bytes' sha256, so other bytes would be another name.
+// It is worked out from the built tree alone, not from the gallery's
+// own list, so the two are compared rather than one copied.
+func fingerprinted(name string, data []byte) bool {
+ m := hashedAsset.FindStringSubmatch(name)
+ if m == nil {
+ return false
+ }
+ sum := sha256.Sum256(data)
+ return hex.EncodeToString(sum[:8]) == m[1]
+}
+
+// versionLess orders X.Y.Z names numerically.
+func versionLess(a, b string) int {
+ var x, y [3]int
+ fmt.Sscanf(a, "%d.%d.%d", &x[0], &x[1], &x[2])
+ fmt.Sscanf(b, "%d.%d.%d", &y[0], &y[1], &y[2])
+ for i := range x {
+ if x[i] != y[i] {
+ return x[i] - y[i]
+ }
+ }
+ return 0
+}
+
+// TestTheSitesHeaders reads the built site's _headers as the edge
+// does, and holds it to what the site needs:
+//
+// - exactly its rules: /M/* for each major, five minutes, since /M/
+// moves to each new M.x; /X.Y.Z/* for each release, a year and
+// immutable, since a published release never changes; both with
+// the cross-origin headers; and each of the design system's hashed
+// assets, a year and immutable, alone. In that order, general
+// before specific.
+// - every file the browser can be served immutable is a release's or
+// a hashed asset, and every hashed asset is immutable; every other
+// page gets no Cache-Control from the file, so the edge's no-cache
+// stays.
+// - every rule's path matches at least one built file.
+// - the file is one `carlos ship` packs, under the edge's limits.
+func TestTheSitesHeaders(t *testing.T) {
+ tree := builtSite(t)
+ data, ok := tree["_headers"]
+ if !ok {
+ t.Fatal("the site has no _headers")
+ }
+ rules, err := parseHeaders(data)
+ if err != nil {
+ t.Fatalf("carlos ship would refuse _headers: %v", err)
+ }
+
+ var versions []string
+ majors := map[string]bool{}
+ var hashed []string
+ for name, body := range tree {
+ first, _, _ := strings.Cut(name, "/")
+ if versionDir.MatchString(first) && !slices.Contains(versions, first) {
+ versions = append(versions, first)
+ major, _, _ := strings.Cut(first, ".")
+ majors[major] = true
+ }
+ if fingerprinted(name, body) {
+ hashed = append(hashed, "/"+name)
+ }
+ }
+ slices.SortFunc(versions, versionLess)
+ slices.Sort(hashed)
+ if !slices.Contains(versions, fiesta.Version) || len(hashed) == 0 {
+ t.Fatalf("the site holds releases %v and %d hashed assets; the checks below need the current release and the gallery's assets", versions, len(hashed))
+ }
+ majorList := make([]string, 0, len(majors))
+ for m := range majors {
+ majorList = append(majorList, m)
+ }
+ slices.SortFunc(majorList, func(a, b string) int { return versionLess(a+".0.0", b+".0.0") })
+
+ t.Run("exactly its rules", func(t *testing.T) {
+ type want struct {
+ path string
+ set http.Header
+ }
+ var wants []want
+ for _, m := range majorList {
+ wants = append(wants, want{"/" + m + "/*", withCache(crossOrigin, fiveMinute)})
+ }
+ for _, v := range versions {
+ wants = append(wants, want{"/" + v + "/*", withCache(crossOrigin, yearLong)})
+ }
+ for _, p := range hashed {
+ wants = append(wants, want{p, withCache(nil, yearLong)})
+ }
+ for i := 0; i < max(len(rules), len(wants)); i++ {
+ switch {
+ case i >= len(rules):
+ t.Errorf("_headers has no rule for %s", wants[i].path)
+ case i >= len(wants):
+ t.Errorf("_headers line %d: %s is a rule the site does not need", rules[i].line, rules[i].path)
+ case rules[i].path != wants[i].path || fmt.Sprint(rules[i].set) != fmt.Sprint(wants[i].set):
+ t.Errorf("_headers line %d: %s %v; want %s %v", rules[i].line, rules[i].path, rules[i].set, wants[i].path, wants[i].set)
+ }
+ }
+ t.Logf("_headers: %d majors, %d releases, %d hashed assets", len(majorList), len(versions), len(hashed))
+ })
+
+ t.Run("immutable is a release or a hashed asset", func(t *testing.T) {
+ for name, body := range tree {
+ if name == "_redirects" || name == "_headers" {
+ continue
+ }
+ first, _, _ := strings.Cut(name, "/")
+ got := lookupHeaders(rules, "", path.Clean("/"+name))
+ cc := got.Get("Cache-Control")
+ switch {
+ case versionDir.MatchString(first) || fingerprinted(name, body):
+ if cc != yearLong {
+ t.Errorf("/%s gets Cache-Control %q; it never changes, so want %q", name, cc, yearLong)
+ }
+ case len(got) != 0:
+ t.Errorf("/%s gets %v; it can change under its name, so it must keep the edge's defaults", name, got)
+ }
+ }
+ })
+
+ t.Run("every rule matches a built file", func(t *testing.T) {
+ for _, r := range rules {
+ found := false
+ for name := range tree {
+ if _, ok := r.match("", segments(path.Clean("/"+name))); ok && name != "_headers" && name != "_redirects" {
+ found = true
+ break
+ }
+ }
+ // A major's files are its newest release's, through the
+ // rewrite, not files at /M/ itself.
+ if !found && strings.HasSuffix(r.path, "/*") {
+ m := strings.Trim(strings.TrimSuffix(r.path, "*"), "/")
+ for _, v := range versions {
+ if strings.HasPrefix(v, m+".") {
+ found = true
+ }
+ }
+ }
+ if !found {
+ t.Errorf("_headers line %d: %s matches no file the site serves", r.line, r.path)
+ }
+ }
+ })
+
+ t.Run("within the edge's limits", func(t *testing.T) {
+ longest := 0
+ for _, l := range strings.Split(string(data), "\n") {
+ longest = max(longest, len(l))
+ }
+ if len(rules) > 90 {
+ t.Errorf("_headers names %d paths; the build stops at 90, ten short of the edge's %d", len(rules), headersMaxPaths)
+ }
+ t.Logf("_headers: %d paths, %d bytes, longest line %d characters", len(rules), len(data), longest)
+ })
+}
diff --git a/internal/www/serve_test.go b/internal/www/serve_test.go
index 55ceae1..c2dcf81 100644
--- a/internal/www/serve_test.go
+++ b/internal/www/serve_test.go
@@ -38,7 +38,7 @@ func builtSite(t *testing.T) map[string][]byte {
if err != nil {
t.Fatalf("Pages: %v", err)
}
- tree, err := site.Build(os.DirFS("../../releases"), www.FS(pages), www.Redirects(pages)...)
+ tree, err := site.Build(os.DirFS("../../releases"), www.FS(pages), www.Rules(pages))
if err != nil {
t.Fatalf("site.Build: %v", err)
}
diff --git a/internal/www/www.go b/internal/www/www.go
index 38b58c1..7a168e2 100644
--- a/internal/www/www.go
+++ b/internal/www/www.go
@@ -22,6 +22,7 @@ import (
"strings"
"amadan.net/rastrillo/fiesta/internal/gallery"
+ "amadan.net/rastrillo/fiesta/internal/site"
)
// Mount is the URL path the design system is served from. The gallery
@@ -123,12 +124,17 @@ func Pages(src fs.FS, siteDir fs.FS) (map[string][]byte, error) {
return out, nil
}
-// Redirects are the _redirects rules the pages need beside the files:
-// the design system's (gallery.Redirects), which send the addresses
-// its pages had before they moved under a locale to the pages there
-// now. pages is what Pages returned; site.Build writes the rules after
-// the release rewrites.
-func Redirects(pages map[string][]byte) []string {
+// Rules are what the pages ask of the site's rules files, for
+// site.Build: their redirects and their immutable addresses. pages is
+// what Pages returned. The callers take both from here, so a site
+// built one place cannot lose a rule another place keeps.
+func Rules(pages map[string][]byte) site.Rules {
+ return site.Rules{Redirects: Redirects(pages), Immutable: Immutable(pages)}
+}
+
+// designSystem is the part of pages under Mount, keyed by its path
+// there, as internal/gallery rendered it.
+func designSystem(pages map[string][]byte) map[string][]byte {
prefix := strings.TrimPrefix(Mount, "/") + "/"
ds := map[string][]byte{}
for name, data := range pages {
@@ -136,5 +142,21 @@ func Redirects(pages map[string][]byte) []string {
ds[rest] = data
}
}
- return gallery.Redirects(Mount, ds)
+ return ds
+}
+
+// Immutable are the addresses of the design system's hashed assets
+// (gallery.Fingerprinted): a new version of one is a new name, so the
+// bytes at each address never change. pages is what Pages returned.
+func Immutable(pages map[string][]byte) []string {
+ return gallery.Fingerprinted(Mount, designSystem(pages))
+}
+
+// Redirects are the _redirects rules the pages need beside the files:
+// the design system's (gallery.Redirects), which send the addresses
+// its pages had before they moved under a locale to the pages there
+// now. pages is what Pages returned; site.Build writes the rules after
+// the release rewrites.
+func Redirects(pages map[string][]byte) []string {
+ return gallery.Redirects(Mount, designSystem(pages))
}
diff --git a/internal/www/www_test.go b/internal/www/www_test.go
index 55d7126..2f5fb98 100644
--- a/internal/www/www_test.go
+++ b/internal/www/www_test.go
@@ -179,7 +179,7 @@ func releases() fstest.MapFS {
// what Pages wrote is in the site at the same paths beside the release.
func TestSiteBuildAcceptsThePages(t *testing.T) {
out := pages(t, siteOf(map[string]string{"static/x.css": "body{}\n"}))
- built, err := site.Build(releases(), www.FS(out))
+ built, err := site.Build(releases(), www.FS(out), site.Rules{})
if err != nil {
t.Fatalf("site.Build refused the pages: %v", err)
}
diff --git a/releases_test.go b/releases_test.go
index 2936b68..012c350 100644
--- a/releases_test.go
+++ b/releases_test.go
@@ -285,7 +285,7 @@ func TestPublishedReleasesMatchTheirTags(t *testing.T) {
// commit holds, and the current release is served under its exact
// version with a /1/ rewrite onto it.
func TestTheSiteBuildsFromTheReleases(t *testing.T) {
- files, err := site.Build(os.DirFS(releasesDir), os.DirFS("site"))
+ files, err := site.Build(os.DirFS(releasesDir), os.DirFS("site"), site.Rules{})
if err != nil {
t.Fatalf("the site does not build from %s/ and site/: %v", releasesDir, err)
}