| 1 | # Directory roster HTTP and signing integration |
| 2 | |
| 3 | Status: design-only; review record in Home discussion 34. Explicit Fable design |
| 4 | approval is required before implementation. |
| 5 | Scope: Home discussion 34, stacked on directory runtime 171f49a, shared roster |
| 6 | 667dc82 and producer 32fb49c. The approved roster contract (discussion 28) and |
| 7 | producer storage contract (discussion 33) remain authoritative. This design adds |
| 8 | runtime wiring and an HTTP adapter; it adds no operator mutation or provisioning |
| 9 | transport and makes no deployment/readiness claim. |
| 10 | |
| 11 | ## Source audit and boundaries |
| 12 | |
| 13 | `runtime.go` currently merges sessions, handoff and directorydata.Schema only for |
| 14 | the directory role. The credential role alone owns `Runtime.signer` and |
| 15 | HOME_ASSERTION_KEY. Directory configuration holds credential public verification |
| 16 | keys and its own sealer; `newDirectory` rejects a credential private signer. |
| 17 | `directoryRouter` puts sessions middleware around the browser routes. |
| 18 | `directoryFeatures` already supplies the compiled app registry and validated |
| 19 | team-origin policy. Reuse that registry and policy when constructing the producer; |
| 20 | do not duplicate per-team rows in environment configuration. |
| 21 | |
| 22 | The published client builds GET directory+/roster with `url.Values.Encode`, a |
| 23 | Bearer token, Accept application/json and Accept-Encoding identity. It refuses |
| 24 | redirects, compression, bodies over 64 KiB and Content-Type values other than |
| 25 | exact application/json. It checks signatures before using page fields. Page-zero |
| 26 | CursorHash is SHA256(empty), not an empty string. There is no epoch query or 304 |
| 27 | shortcut. Wire members use person_ref. The pending Docs cache lease amendment |
| 28 | changes no Page/SignPage contract; this adapter remains compatible with 667dc82. |
| 29 | |
| 30 | ## Runtime and schema |
| 31 | |
| 32 | Merge directorydata.RosterSchema immediately after directorydata.Schema in the |
| 33 | directory-only schema. Credential and legacy schemas must not gain these tables |
| 34 | or membership triggers. Applying this migration leaves generation, consumers, |
| 35 | tokens and receipts empty. Repeated startup never fills them. |
| 36 | |
| 37 | Introduce HOME_ROSTER_MODE with absent/disabled and serve values. In directory |
| 38 | mode, absent/disabled plus an absent HOME_ROSTER_KEY means a disabled endpoint. |
| 39 | The directory browser routes and boot remain available; every request that passes |
| 40 | the roster host/method guards receives the same fixed 503 before inspecting any |
| 41 | token, query or database. There is no standalone roster fallback. |
| 42 | |
| 43 | Serve requires HOME_ROSTER_KEY. Unknown mode, key supplied while disabled, missing |
| 44 | serve key or invalid key refuses startup with one fixed configuration error. |
| 45 | Credential and legacy roles reject either roster environment variable when set, |
| 46 | including an explicit disabled value. Direct Runtime constructors enforce the |
| 47 | same mode/key invariants; LoadRuntime is not the only guard. |
| 48 | |
| 49 | Represent roster configuration separately, for example a private rosterRuntime |
| 50 | field with mode, signer and derived key ID. Keep Runtime.signer credential-only |
| 51 | and preserve newDirectory's existing rejection of it. Directory now deliberately |
| 52 | owns one new roster-only private key; update role comments, validation and tests |
| 53 | that currently equate all private signing keys with credential authority. |
| 54 | |
| 55 | HOME_ROSTER_KEY is canonical standard-base64 of the existing family.MarshalKeypair |
| 56 | format, bounded to 4096 encoded bytes, with no alternate file or second encoding |
| 57 | input. Decode strictly and verify the encoding round trip. Validate the closed |
| 58 | keypair JSON before calling family.UnmarshalKeypair: exactly sign_priv_d and |
| 59 | box_priv_d, canonical 64-character lowercase hexadecimal strings, no duplicate or |
| 60 | unknown fields. Require P256 signing scalar 0<D<N and valid box scalar; derived |
| 61 | public keys come from the existing library. The box half is unused by roster. |
| 62 | Derive roster key ID as lowercase hex SHA256(SignPub()), matching the shared |
| 63 | 64-byte atom bound. Do not take the key ID from request input. |
| 64 | |
| 65 | Reject a roster signing public key equal to any configured credential assertion |
| 66 | verification key. The independently audited operator provisioning key must also |
| 67 | be distinct; it is not loaded by this producer and its separation is a writer/ |
| 68 | provisioning gate. Never derive a roster private key from HOME_DEK, pepper, token, |
| 69 | credential key or startup randomness. Startup does not publish a verification key |
| 70 | or manufacture a signed operator record. Key rotation requires the separately |
| 71 | approved operator record/key rollout before changing the active signing key. |
| 72 | |
| 73 | ## Handler seam and dispatch |
| 74 | |
| 75 | Use an unexported rosterHandler with the concrete validated roster runtime, |
| 76 | configured directory origin, clock seam and a small read-source interface: |
| 77 | Read(context.Context, directorydata.RosterRequest, time.Time) |
| 78 | (roster.Page, int, error). Production injects NewRosterSource using the existing |
| 79 | store's DB/sealer/app policy. Tests may inject a counting source or real store. |
| 80 | Disabled mode does not construct or invoke a source or signer. |
| 81 | |
| 82 | Dispatch decoded URL.Path == /roster to this adapter before directory browser |
| 83 | session/chooser middleware. The adapter owns all methods for that exact route, |
| 84 | so chi's default method response cannot bypass its headers. Cookies neither |
| 85 | authorize the request nor cause Set-Cookie, sign-in redirects or session writes. |
| 86 | All other routes retain the existing directory router and guards. The roster |
| 87 | adapter does not reuse browser CSRF or assertion-consumer middleware. |
| 88 | |
| 89 | At adapter entry set Cache-Control: no-store, no-transform, Content-Type: application/json and |
| 90 | X-Content-Type-Options: nosniff. No CORS permission, compression, ETag, Last-Modified, |
| 91 | Location or browser cookie is emitted. Refusal precedence is: |
| 92 | |
| 93 | 1. r.Host must equal the configured directory host, including any configured |
| 94 | port, exactly. No trailing-dot, alternate-host or forwarded-header alias. |
| 95 | Wrong host returns fixed 404, before source access. |
| 96 | 2. Method must be GET; HEAD, POST, OPTIONS and all others return fixed 405 with |
| 97 | Allow: GET. Nothing follows a redirect to turn a POST into a roster read. |
| 98 | 3. Disabled runtime returns fixed 503 for every token/query variation. |
| 99 | 4. Validate request syntax below; malformed query/path/body returns fixed 400. |
| 100 | 5. Validate one exact bearer header; malformed/missing/duplicate returns fixed401. |
| 101 | 6. Call the source once with a current server time. Nonpositive server time |
| 102 | returns fixed503 here, before the source's own input check. |
| 103 | |
| 104 | Require URL.EscapedPath() exactly /roster, no absolute request-target, no fragment, |
| 105 | and no nonempty GET body: ContentLength must be zero and TransferEncoding empty. |
| 106 | Do not read an attacker-sized body. The configured directory origin plus the |
| 107 | request path/query must fit 4096 bytes; bound RequestURI too. A valid first query |
| 108 | contains exactly team and fetch. A continuation additionally contains one |
| 109 | nonempty cursor. Query field order is unrestricted. Reject duplicate/unknown or |
| 110 | empty fields, extra separators, semicolons, percent encodings, plus encodings and |
| 111 | padding. Parse literal bounded name=value pairs; do not use a helper that silently |
| 112 | discards malformed parameters. Team is 1..64 ASCII alphanumerics, fetch passes |
| 113 | roster.ValidChallenge, and cursor is strict canonical raw-base64url at most |
| 114 | roster.CursorLimit (2048 bytes), checked by decoding and round trip. The producer |
| 115 | validates the sealed cursor contents again. Neither team nor cursor selects a |
| 116 | network destination. |
| 117 | |
| 118 | Count Authorization values across case-insensitive header names; require exactly |
| 119 | one value equal to Bearer + one ASCII space + a canonical43-character token. |
| 120 | Reject comma lists, repeated lines, tabs, extra spaces, alternate schemes and |
| 121 | padded/noncanonical values. Use roster.ValidChallenge before calling the source. |
| 122 | No browser session, Origin, Referer or query token substitutes for this header. |
| 123 | Accept and Accept-Encoding need no special value from the caller: the adapter |
| 124 | always returns the one JSON representation without compression. The no-transform |
| 125 | directive is required by the CARLOS edge compressor as well as the app adapter: |
| 126 | without it, application/json responses can be gzipped for Accept-Encoding: gzip, |
| 127 | including small error responses when the upstream length is unknown. |
| 128 | |
| 129 | The application adapter never redirects. The existing outer Go ServeMux can |
| 130 | canonicalize unrelated malformed paths before the application sees them; the |
| 131 | shared client never follows these redirects. Tests through the real outer mux |
| 132 | must show no alternate path produces a trusted page, and canonical /roster never |
| 133 | redirects for any method/query/error. No change to global router normalization is |
| 134 | needed to serve the fixed canonical endpoint. |
| 135 | |
| 136 | ## Response and error boundary |
| 137 | |
| 138 | Give source.Read a request-derived context capped at five seconds, and obtain |
| 139 | now immediately before it. No network call or operator write occurs under that |
| 140 | read transaction. On nil error, accept only status200 with type page or status409 |
| 141 | with type stale-cursor. Sign with roster.SignPage using the validated roster key |
| 142 | and derived ID, and buffer the complete returned bytes before writing headers. |
| 143 | Check len <= roster.BodyLimit as a final boundary. Sign/marshal/status failure |
| 144 | returns the fixed unsigned503; never emit an unsigned Page or a partial signed |
| 145 | response. Do not duplicate the shared envelope/signature or JSON-pointer logic. |
| 146 | |
| 147 | A source failure maps only its fixed401/404/400/503 error/status pairs. Unexpected |
| 148 | pairs become fixed503. Keep the approved source's exact mapping, including401 |
| 149 | for a missing token-to-registry join; do not infer a more revealing cause. |
| 150 | Authenticated epoch/generation change is the signed409 shape, with current scope, |
| 151 | fetch, request-cursor hash, generation and epoch and all page-only fields absent. |
| 152 | It is never treated as an empty successful roster or a new generation approval. |
| 153 | |
| 154 | Unsigned errors contain only a fixed machine code: bad_request, unauthorized, |
| 155 | not_found, method_not_allowed or unavailable in an error JSON member. No supplied |
| 156 | value or underlying error is serialized. No response includes a team name, email, |
| 157 | origin selected from request input or token diagnostic. All errors keep no-store |
| 158 | and exact application/json, although the client still refuses unsigned errors. |
| 159 | No raw URLs, bodies, cursor, token, label, subject or key enters logs, traces, |
| 160 | metrics or status messages. The handler itself needs no logging. Transport and |
| 161 | platform access-log behavior must be checked before any live enablement; the |
| 162 | application's no-log claim does not attest to an unaudited proxy configuration. |
| 163 | |
| 164 | ## Authority and deployment gates |
| 165 | |
| 166 | In serve mode an otherwise authenticated, scoped request with absent generation |
| 167 | returns503 through the source; |
| 168 | it must not mint a generation, owner, membership, token or receipt. Invalid or |
| 169 | unknown tokens remain401 and foreign/unknown teams remain identical404. Merely |
| 170 | installing the key does not initialize or approve authority. Runtime health may |
| 171 | report process health as before; it is not a claim that roster provisioning is |
| 172 | complete or that Docs TeamMode is ready. |
| 173 | |
| 174 | The future writer must authenticate independent operator authority before its |
| 175 | transaction, verify the exact scope/expected generation/record sequence, assert |
| 176 | foreign_keys ON, and atomically write state and a successful receipt. It must |
| 177 | follow the producer storage contract's token-slot and epoch-trigger rules. |
| 178 | This slice adds no such writer, route, command, migration seed or environment |
| 179 | import. No first visitor or roster bearer can supply it. Existing-db bootstrap |
| 180 | and restore remain behind the supported fenced maintenance path (platform117); |
| 181 | a genuinely fresh first-boot path needs separate audit. Without either approved |
| 182 | path there is no live generation/consumer initialization and no general team |
| 183 | roster launch. Test SQL and test operator signatures prove behavior only. |
| 184 | |
| 185 | ## Implementation acceptance |
| 186 | |
| 187 | Run the complete Home gate, focused producer/handler race tests, actual CI and |
| 188 | Fable whole-code review on a frozen candidate before parent integration. Tests |
| 189 | must use actual runtime/schema/router paths as well as the injected handler: |
| 190 | |
| 191 | - Directory schema contains roster tables/triggers; credential/legacy schemas do |
| 192 | not. Empty and repeated startup leave authority tables empty. Disabled runtime |
| 193 | returns identical503 across token/query inputs without DB/signing/session work. |
| 194 | - Mode/key matrix covers both environment loader and direct constructors, role |
| 195 | separation, malformed/private-key scalar/JSON cases, credential key reuse and |
| 196 | no startup generation. Existing Home handoff/chooser browser gates still pass. |
| 197 | - Exact Host, all wrong methods, duplicate/mixed-case Authorization maps, comma/ |
| 198 | whitespace forms, absent or noncanonical challenges, hostile team/cursor, |
| 199 | duplicate/unknown/escaped query keys, empty cursor, oversized target and GET |
| 200 | body all refuse before any source call. Errors leak none of the test corpus. |
| 201 | - Use real directory fixture transactions and the actual shared Cache client |
| 202 | against the handler through an injected same-origin TLS test transport. Signed |
| 203 | 200 with zero, one and more than100 members verifies; exact empty array stays |
| 204 | explicit. Rotation, disable and foreign/unknown team failures remain bounded. |
| 205 | - Between pages mutate membership or generation; verify signed409 and no cache |
| 206 | publication/lease extension. Wrong key/context/scope/signature cannot become |
| 207 | standing. A missing generation never signs a successful page. No epoch/304 |
| 208 | response path exists. Two consumers and two teams stay isolated. |
| 209 | - Buffer/sign failures return unsigned503 with no partial Page. Every adapter |
| 210 | result has no-store, no-transform/exact JSON and no Location, encoding, validator or cookie. |
| 211 | The outer mux/real server retains these for canonical /roster, and no redirect |
| 212 | is followed by the shared client. |
| 213 | - Seed names, tokens and cursor plaintext in fixtures and scan DB/WAL plus |
| 214 | captured application logs. Negative mutations removing bearer/mode/Host |
| 215 | checks, bypassing SignPage, or dropping signed409 expose failing tests. |
| 216 | |
| 217 | No browser UI is introduced by this endpoint. A deployed two-team transport |
| 218 | canary is a later root-owned gate once an audited provisioning path exists; this |
| 219 | design and local fixtures do not authorize live seeding or mark the suite ready. |
| 220 | |
amadan