| 1 | # Nearest region by default: each app picks its region at creation |
| 2 | |
| 3 | **Date:** 2026-10-01 |
| 4 | **Revision:** 6.2. |
| 5 | |
| 6 | - Rev 1 (7fbc9479) went to Astra (GPT-6, via codex), and the findings are in |
| 7 | `CODEX-FINDINGS-NEAREST-REGION-R1.md`. |
| 8 | - Rev 2 (bd4ed0c4) went back, and the findings are in |
| 9 | `CODEX-FINDINGS-NEAREST-REGION-R2.md`. |
| 10 | - Rev 3 (8e59f598) went back again, and the findings are in |
| 11 | `CODEX-FINDINGS-NEAREST-REGION-R3.md`. |
| 12 | - Every finding of all three rounds was checked against the code and |
| 13 | accepted. Changes are marked "(codex N)", "(codex R2-N)" and |
| 14 | "(codex R3-N)", and §21 records each disposition. |
| 15 | - Rev 3 folded in Paul's answers to rev 2's questions as rulings R5 to R7. |
| 16 | - **Rev 4 moves latency routing (R6) into a follow-on spec**, because the |
| 17 | existing steering converger is not safe to arm (§13). That spec gets |
| 18 | invariants to meet, and placement ships first. |
| 19 | - Rev 4.1 records Paul's ruling on that split as R8 and removes the question. |
| 20 | - Rev 5 answers Astra's round 4 (`CODEX-FINDINGS-NEAREST-REGION-R4.md`): three |
| 21 | P2s, marked "(codex R4-N)". Astra's round 5 found nothing new; Paul |
| 22 | approved rev 5 on 2026-10-02. |
| 23 | - **Rev 6 is an amendment Paul approved on 2026-10-02, ruling R9**, after a |
| 24 | production finding: only the Ireland box holds `CARLOS_SECRETS_KEY`. On the |
| 25 | UK, US and AU boxes the config loops log "config sync loop disabled: |
| 26 | CARLOS_SECRETS_KEY not set" (`internal/host/agent.go:881-886`), and a wake |
| 27 | treats a missing `<host>.config.env` as empty |
| 28 | (`internal/activator/backend_exec.go:1146-1149`). An instance placed there |
| 29 | runs with no config, no secrets and no store binding. That was part of the |
| 30 | kpmg outage. Changes are marked "(R9)". |
| 31 | - **Rev 6.1 records ruling R9a** (Paul, 2026-10-02): `app_default` gains |
| 32 | `region_known` and `placement_known`. They are additive, and absent |
| 33 | knowledge fails closed. They came out of the implementation plan's review |
| 34 | (Astra plan round 2, finding 3), and §9.3 states their semantics. |
| 35 | - **Rev 6.2 records `instance_domain`** (Paul approved it on 2026-10-03): |
| 36 | an additive, read-only field of `app_default`, the authoritative hostname |
| 37 | suffix `canary --region` reads its own record under before any write |
| 38 | (§9.3). It came out of Astra's plan round 3 (finding 1), and her round 4 |
| 39 | judged it sound. Rev 6.2 also writes down a consequence of R9a that round |
| 40 | 4 found (finding 1): with the stored region unread, `effective` is empty |
| 41 | even for a pooled or fleet-placed app, so `canary --region` needs both |
| 42 | flags. |
| 43 | |
| 44 | **Status:** Approved by Paul on 2026-10-02: rev 5, plus the R9 amendment in |
| 45 | this revision. The rulings in §3 are Paul's and are not reopened here. |
| 46 | **Discussion:** not yet filed on amadan. Latency routing depends on amadan |
| 47 | #169. |
| 48 | **Builds on:** |
| 49 | |
| 50 | - #329, the armed-placement gate |
| 51 | - #346, the home region's name (`CONSOLE_HOME_REGION`) |
| 52 | - the customer-fleets placement column |
| 53 | - #162, replica placements |
| 54 | - #175/#331, instance moves |
| 55 | - the multi-region steering converger (`internal/steering`) |
| 56 | |
| 57 | ## 1. The problem, as found |
| 58 | |
| 59 | On 2026-10-01 the Ireland box carried 116 of the flagship's 116 non-static |
| 60 | instances and reached 91% disk. Overnight, one customer's three instances |
| 61 | crash-looped for 4h18m on a full disk. Nothing chose Ireland. Nothing chose |
| 62 | anything. |
| 63 | |
| 64 | Every create path ends at one door, and that door's default is the home box: |
| 65 | |
| 66 | - The instance create door, `handleAPIInstanceCreate` |
| 67 | (`internal/console/apiinstances.go:608`), trims `req.Region` and hands it to |
| 68 | `homeRegionIn` (`apiinstances.go:772`; the function is at |
| 69 | `internal/console/apiinstancemove.go:136`). The home's name normalizes to |
| 70 | `""`, and `""` is the unconstrained record only the home box mints. |
| 71 | - The new-app page (`internal/console/templates/app_new.html`) asks for a name |
| 72 | and nothing else. The Instances tab's Placement section |
| 73 | (`templates/app_instances.html:60-78`) offers "Shared fleet" or one of the |
| 74 | account's own fleets, which is a different axis. |
| 75 | - `carlos deploy` creates an app only through `offerCreateApp` |
| 76 | (`deploy.go:513`, call at `:536`), and sends no region. Its only instance |
| 77 | write is the static platform host (`mintPlatformHost`, `deploy.go:850`, |
| 78 | create at `:858`), which sends `""`. |
| 79 | - `carlos canary` creates its host at `canary.go:407` with region `""`. |
| 80 | - Only `carlos instances create --region` (`instances_cli.go:293`, sent at |
| 81 | `:322`) chooses, and only when a person remembers to. External signup paths |
| 82 | (titogo, Managed Jelly) hold instance-role tokens and send no region. |
| 83 | |
| 84 | And nothing is steered. Latency routing exists (`internal/steering`), but it |
| 85 | is armed nowhere, because no deployment sets `CARLOS_STEERING_ZONE_ID` |
| 86 | (amadan #169). Every instance host therefore resolves through the |
| 87 | deployment's static DNS. On the flagship that is one `*.oncarlos.com` record |
| 88 | pointing at the Ireland EIP, served by DNSimple. The Route53 copy of the zone |
| 89 | is staged and is not yet authoritative (carloku-infrastructure `dns.tf` and |
| 90 | `dns53.tf`). |
| 91 | |
| 92 | What exists to build on: |
| 93 | |
| 94 | - **Placement codes** are bare (`ie`, `uk`, `au`): `CARLOS_REGION` verbatim. |
| 95 | - **Place names**: the only one anywhere is `STATUS_BOX_REGION`, operator free |
| 96 | text for the public status page (`internal/host/agent.go:88-92`). |
| 97 | - **The edges file**: every box publishes its AWS region and its placement |
| 98 | code there (`internal/host/edgesfile.go:155-176`). The region comes from |
| 99 | `edgesFileRegion` (`:537`), which honours a `CARLOS_AWS_REGION` override. |
| 100 | - **What the console already reads**: both of those, through |
| 101 | `edgesFilePlacementView` (`internal/console/instancesplacement.go:35-55`) |
| 102 | and `servedBy.Region` (`:25-28`). |
| 103 | - **Capability**: `placements` and `armedPlacements` |
| 104 | (`instancesplacement.go:173`, `:207`) say which codes can host. |
| 105 | - **The regions endpoint**: `GET /api/cli/apps/{account}/{app}/regions` |
| 106 | (`internal/console/apiregions.go:31-55`) returns codes, `armed` and |
| 107 | `"default": HomeRegion`, and reads no manifest. |
| 108 | |
| 109 | There is no geolocation of any kind. |
| 110 | |
| 111 | ## 2. Goals and non-goals |
| 112 | |
| 113 | Goals: |
| 114 | |
| 115 | - A new app gets a region at creation, preselected from a guess of where its |
| 116 | creator is. Its new instances start there unless told otherwise. |
| 117 | - A new instance placed outside the home is reached through the nearest edge, |
| 118 | not through the home's (R6). The follow-on spec for Phase L delivers this |
| 119 | goal, against the invariants in §13. |
| 120 | - The guess is generic: any deployment, any set of AWS regions. |
| 121 | - The guess can never name a region the create door would refuse, or one no |
| 122 | box would mint (codex 1, codex R2-1). |
| 123 | - Existing apps, older CLIs and external callers that send nothing behave |
| 124 | exactly as today. |
| 125 | |
| 126 | **Done** has two stages: |
| 127 | |
| 128 | - **This spec** is done when new apps get regions and new instances follow |
| 129 | them. |
| 130 | - **The work as a whole** (R5, R6) is done when the follow-on spec has made |
| 131 | latency routing safe to arm, and #169 is armed. |
| 132 | |
| 133 | Existing apps keep their region. |
| 134 | Relieving Ireland of the instances it already holds is the instance-move |
| 135 | work (#175/#331), not this spec. |
| 136 | |
| 137 | Non-goals, each with a follow-up in §20: |
| 138 | |
| 139 | - Arming DNS steering and changing the converger. This is Phase L, the |
| 140 | follow-on spec (§13). |
| 141 | - Moving existing instances. |
| 142 | - Changing an existing app's region (R5). |
| 143 | - A browser instance-create form. |
| 144 | - Sidecars following the app's region. |
| 145 | - Pool-aware candidates. |
| 146 | - Box sizing and disk alerts. |
| 147 | - An account-level default (R1). |
| 148 | |
| 149 | ## 3. Rulings (Paul, 2026-10-01) |
| 150 | |
| 151 | | # | Ruling | Where this spec spends it | |
| 152 | | --- | --- | --- | |
| 153 | | R1 | Region is both latency and placement, chosen **per app**. Each app picks its region at creation, preselected from a location guess. No account-level default. | §5, §8.1, §13 | |
| 154 | | R2 | The app's region is the **default** for new instances. It can be overridden per instance (picker or `--region`), subject to the manifest `Regions` allow-list and the armed/hosting check. | §7 | |
| 155 | | R3 | Guess **timezone first**. Fall back to IP geolocation when the timezone is UTC, `Etc/*` or otherwise unmappable. | §6 | |
| 156 | | R4 | The CLI guesses too. `carlos deploy` creating an app sends its timezone, the console resolves it, and the CLI prints one line ("Region: UK · London (from your timezone). Use --region to choose another."). No prompt. `--region` overrides everywhere. `carlos instances create` defaults to the app's region. | §11 | |
| 157 | | R5 | **Done = new apps only, plus moves.** Existing apps keep their region. Relieving Ireland comes from instance moves. An owner-only verb to change an existing app's region is out of scope (a follow-up). | §2, §20 | |
| 158 | | R6 | **Latency routing is in scope** for non-home instances, not placement alone. | §13: its invariants and gates, delivered by the follow-on spec | |
| 159 | | R7 | **The rollback floor is a runbook rule.** It is documented for release notes and the roll recipe, with no outside guard. | §5.7, §17 | |
| 160 | | R8 | **Split** (Paul, 2026-10-01). Placement ships on this spec. Latency routing (R6) is sequenced to its own follow-on spec, built to the §13 invariants. | §13, §17 | |
| 161 | | R9 | **A box must hold the deployment's secrets key to receive placed instances** (Paul, 2026-10-02). | §6.1, §7, §8.1, §17 | |
| 162 | | R9a | **`app_default` gains `region_known` and `placement_known`** (additive; absent knowledge fails closed) (Paul, 2026-10-02). | §9.3 | |
| 163 | | D1 | The app region is stored console-side and set once at creation. The create door fills an empty `req.Region` from it **before** normalization, the allow-list and the armed checks. Existing apps are backfilled to the home, so behaviour does not change. | §5, §7 | |
| 164 | | D2 | The guess works in coordinates: timezone to a point, IP to a point through an operator-supplied MaxMind-format `.mmdb` (`CONSOLE_GEOIP_DB`), then the great-circle nearest eligible region. Home when everything fails. | §6 | |
| 165 | | D3 | Surfaces: a new-app picker; names and the guess for external pickers; the CLI as R4. | §9, §10, §11 | |
| 166 | | D4 | Place names come from one platform-shipped AWS-region table, used everywhere and reconciled with `STATUS_BOX_REGION`. | §14 | |
| 167 | |
| 168 | ## 4. The shape, in one paragraph |
| 169 | |
| 170 | The console's `apps` row gains two columns: |
| 171 | |
| 172 | - `region`, in the record's own vocabulary, where `""` is the home; |
| 173 | - `region_source`, one word saying how it was picked. |
| 174 | |
| 175 | Both app-create doors resolve a region before the row is written. The order |
| 176 | is an explicit choice, then the creator's timezone, then the request's IP |
| 177 | (through the optional GeoIP database), then the home. Resolving means |
| 178 | finding the great-circle nearest region whose boxes have positively said |
| 179 | they can mint an ordinary instance. |
| 180 | |
| 181 | From then on, an instance create that names no region gets the app's region, |
| 182 | and that value meets every check an explicit one meets. An **app-derived** |
| 183 | value additionally needs that positive eligibility, while an explicit one |
| 184 | keeps today's gate (codex R3-6). |
| 185 | |
| 186 | Latency routing for non-home instances (R6) is specified as invariants |
| 187 | (§13) and delivered by a follow-on spec. Until that lands, this spec writes |
| 188 | nothing to DNS and stamps nothing. |
| 189 | |
| 190 | Existing rows read `""`, which is what they have always meant. Nothing writes |
| 191 | a non-home app region until the operator turns the feature on, and the |
| 192 | console will not turn it on without a named home. |
| 193 | |
| 194 | ## 5. Data model |
| 195 | |
| 196 | ### 5.1 Columns |
| 197 | |
| 198 | **(decision) Two columns, appended to the ALTER list in `migrate`** |
| 199 | (`internal/console/store.go:91`, list ending `:386`): |
| 200 | |
| 201 | ```sql |
| 202 | ALTER TABLE apps ADD COLUMN region TEXT NOT NULL DEFAULT ''; |
| 203 | ALTER TABLE apps ADD COLUMN region_source TEXT NOT NULL DEFAULT ''; |
| 204 | ``` |
| 205 | |
| 206 | - `region` is a placement code in the **record vocabulary**: `""` is the |
| 207 | deployment's home, exactly as on a `provision.Record` (#346). A choice of |
| 208 | the home's name is normalized with `homeRegionIn` before storing, and |
| 209 | displays use `homeRegionOut` (`apiinstancemove.go:143`). |
| 210 | - `region_source` is `chosen`, `timezone`, `ip`, `default`, or `""` (the row |
| 211 | predates this feature). It is a label for people and logs, and nothing |
| 212 | branches on it. |
| 213 | |
| 214 | **Nothing else is stored**: not the timezone, not the IP, not a coordinate. |
| 215 | |
| 216 | ### 5.2 The rebuild trap |
| 217 | |
| 218 | `migrateAppsKey` (`store.go:472`) rebuilds `apps` with a fixed column list |
| 219 | (`:487-505`) and runs **after** the ALTER list. On a database not yet rebuilt, |
| 220 | the new columns would be added and then dropped by the rebuild's |
| 221 | `INSERT … SELECT`, and every later read of `region` would fail. |
| 222 | |
| 223 | **Both columns go into `apps_new`'s DDL and the INSERT's column list in the |
| 224 | same change.** This is the rule the comment at `:499-503` already records for |
| 225 | `placement`, `control_plane` and `build_probe`. |
| 226 | |
| 227 | ### 5.3 Written with the row |
| 228 | |
| 229 | **(decision) The region rides the INSERT that claims the name.** |
| 230 | |
| 231 | - A new `CreateAppInRegion(accountID, app, createdBy, region, source, now)` |
| 232 | carries both columns. |
| 233 | - `CreateAppIn` (`store.go:1349`) becomes a call to it with `("", "")`, so |
| 234 | `CreateApp` (`:1342`) is unchanged. |
| 235 | - `claimApp` (`internal/console/server.go:3466`) gains a `regionReq` |
| 236 | parameter (§5.5). |
| 237 | |
| 238 | This departs from the placement precedent on purpose. `SetAppPlacement` after |
| 239 | `CreateAppIn` (`server.go:3550-3567`) is best-effort, because `carlos apps |
| 240 | place` can repair it. An app region has no repair verb (R5), so a failed |
| 241 | second write would leave the app at the home for good, silently. |
| 242 | |
| 243 | Rename (`store.go:1791`) and transfer (`:1709`) update the same row, so the |
| 244 | region survives both. The legacy owners backfill (`:665`) inserts defaults, |
| 245 | which mean the home. |
| 246 | |
| 247 | ### 5.4 Backfill |
| 248 | |
| 249 | **(decision) The column default is the backfill.** Every existing row reads |
| 250 | `""`/`""`. `""` is the record vocabulary's home, so the door's fill (§7) turns |
| 251 | `""` into `""` and nothing changes, byte for byte. |
| 252 | |
| 253 | Writing the home's code instead would be worse on every count: |
| 254 | |
| 255 | - It would gain nothing, because `homeRegionIn` normalizes it back to `""`. |
| 256 | - It would be impossible where `CONSOLE_HOME_REGION` is unset. |
| 257 | - It would freeze the home as of migration day, where `""` follows the home |
| 258 | as records already do. |
| 259 | |
| 260 | The backfill is idempotent by construction: the ALTERs swallow "duplicate |
| 261 | column", and there is no one-shot `PRAGMA user_version` step. |
| 262 | |
| 263 | ### 5.5 Explicit choice versus guess, and adoption (codex 2, codex R2-3, codex R2-5) |
| 264 | |
| 265 | `claimApp` receives a request-local struct, not a resolved region: |
| 266 | |
| 267 | ```go |
| 268 | type regionReq struct { |
| 269 | explicit string // a person's choice: a code or the home's name; "" = none |
| 270 | tz string // a hint |
| 271 | ip net.IP // from s.clientIP, set only for a person's request |
| 272 | } |
| 273 | ``` |
| 274 | |
| 275 | **Only `explicit` is a choice.** The guess is resolved inside `claimApp`, |
| 276 | after it knows whether this is an adoption, so no caller can pass a guess off |
| 277 | as a choice. |
| 278 | |
| 279 | `ip` comes from `s.clientIP` (`internal/console/authcli.go:876`). `claimApp` |
| 280 | has no `*http.Request`, and its `addr` argument is the audit actor |
| 281 | (`apicli.go:349`), not a network address (codex R2-5). Both callers fill `ip` |
| 282 | before the call. It lives only for that call, and is never logged or stored. |
| 283 | |
| 284 | **(decision) Adoption never changes an existing app's region by guessing.** |
| 285 | An owner claiming a name whose objects already exist (`adoptingExistingApp`, |
| 286 | `server.go:3502`, `:3545`) is restoring an app that probably has instances at |
| 287 | the home. An adoption stores `""` with source `default` unless `explicit` is |
| 288 | set. A guess, however it was computed and whatever the form displayed, is |
| 289 | discarded. |
| 290 | |
| 291 | The browser form keeps the two apart without trusting any label. The |
| 292 | preselected option's value is `@auto`, which is outside the placement-code |
| 293 | grammar (`^[a-z0-9-]{0,16}$`, `provision.go:688`). A deployment with a |
| 294 | region, or a home, named `auto` therefore still works (codex R2-3). Only a |
| 295 | region code is `explicit` (§10). The CLI sends `region` only for `--region`, |
| 296 | and sends `timezone` separately (§9.1). |
| 297 | |
| 298 | ### 5.6 The switch and the named home (codex 3, codex 4) |
| 299 | |
| 300 | **(decision) Non-home app regions are written only when both of these hold:** |
| 301 | |
| 302 | - `CONSOLE_APP_REGIONS=on` (new; off by default); |
| 303 | - `CONSOLE_HOME_REGION` is set. |
| 304 | |
| 305 | If the switch is on and the home is unnamed, the console logs that once at |
| 306 | startup and behaves as if the switch were off. |
| 307 | |
| 308 | Without a named home, `""` is the only way to say "home", and `""` also means |
| 309 | "no preference". No request could then choose the home over a non-home app |
| 310 | default. A named home makes `"region": "<home name>"` an explicit choice at |
| 311 | every door. |
| 312 | |
| 313 | While the switch is effectively off: |
| 314 | |
| 315 | - Every new app gets `""`/`default`. An explicit non-home `region` on app |
| 316 | create is not stored, and the response shows what was (§9.1). |
| 317 | - The new-app picker is hidden. |
| 318 | - The guess endpoint (§9.4) still answers, because a per-instance guess for |
| 319 | titogo writes nothing to the app row. |
| 320 | - **The door's fill still runs.** Turning the switch off stops new promises. |
| 321 | It never breaks the ones already made. |
| 322 | |
| 323 | ### 5.7 Mixed versions and rollback (codex 4, R7) |
| 324 | |
| 325 | The console owns its SQLite database, so one database is served by one |
| 326 | console at a time. Mixed versions therefore means a rollback, or a roll that |
| 327 | has not yet reached a deployment's console. A console older than this feature |
| 328 | ignores the columns, so after a rollback it would land a no-region create for |
| 329 | a `uk` app at the home. |
| 330 | |
| 331 | **(decision, under R7) The rule:** |
| 332 | |
| 333 | 1. **Roll the read side first.** The first release carries the migration, the |
| 334 | fill, the endpoints and the UI, with the switch off. Every app row is then |
| 335 | `""`, so the release can be rolled back freely. |
| 336 | 2. **The switch is the feature's rollback lever.** It is turned on only after |
| 337 | that release has run cleanly. Turning it off stops new non-home values and |
| 338 | keeps the fill. |
| 339 | 3. **Once any app holds a non-home region, the binary rollback floor is the |
| 340 | first region-aware release.** This is a runbook rule (R7), with no outside |
| 341 | guard. It is written in three places: |
| 342 | - the README's "Releasing the platform" section (`README.md:227`); |
| 343 | - that release's notes; |
| 344 | - the console roll recipe in `carlosframework/carloku-infrastructure`. |
| 345 | |
| 346 | Rolling below the floor is out of policy. If it happens anyway, no-region |
| 347 | creates land at the home, as they do today. Nothing is lost. |
| 348 | 4. **On roll-forward, the console logs possible placement discrepancies** |
| 349 | (codex R2-6). For each app with a non-home region, it lists records that |
| 350 | carry an empty region and were created after the app. |
| 351 | |
| 352 | Records that never inherit the app default are excluded: static, |
| 353 | addr-backed, replica, pooled and fleet-placed. Those are the shapes §7 |
| 354 | leaves unfilled. A deliberate home override also produces an empty |
| 355 | region, and the record keeps no provenance, so the list is a set of |
| 356 | candidates for a person to review, not a detection of rollback damage. |
| 357 | The runbook records the rollback interval, so the review can be narrowed |
| 358 | to records created inside it. |
| 359 | |
| 360 | CLIs mix freely in both directions (§9.6). |
| 361 | |
| 362 | ## 6. Resolving a region |
| 363 | |
| 364 | ### 6.1 Candidates and eligibility (codex R2-1) |
| 365 | |
| 366 | **(decision) One function, `regionCandidates(ctx, policy)`, in |
| 367 | `instancesplacement.go`, built from the same edges-file read `placements` |
| 368 | uses.** Each candidate is `{code, awsRegion, eligible}`. |
| 369 | |
| 370 | **Eligibility is positive and per box.** `Hosting` checks reachability and |
| 371 | the activator, but not whether a box will mint an *ordinary* instance: |
| 372 | |
| 373 | - A box with `CARLOS_POOLS` set serves the shared pool only if the list names |
| 374 | it (`poolServed`, `internal/provision/reconcile.go:1339`). |
| 375 | - A box with `CARLOS_INBOUND=none` refuses every routed instance |
| 376 | (`reconcile.go:925`). |
| 377 | |
| 378 | Neither fact is in the edges file today. **(decision) Each box publishes two |
| 379 | additive fields** in `edgesFile` (`internal/host/edgesfile.go:155-176`): |
| 380 | |
| 381 | - `serves_shared_pool`: `poolServed(cfg.Pools, "")`; |
| 382 | - `accepts_inbound`: `cfg.Inbound != InboundNone`. |
| 383 | |
| 384 | **Both are computed from the very `provision.ReconcileConfig` the reconcile |
| 385 | pass uses**, which `provisionTickConfig` builds (`internal/host/agent.go:2030`). |
| 386 | They are never re-parsed from the environment (codex R3-3). That parser |
| 387 | drops empty elements, so `CARLOS_POOLS=,tito` is `["tito"]` and does **not** |
| 388 | serve the shared pool. The box publishes `false` for it. A box that should |
| 389 | serve both the shared pool and a named one needs a parser change with its |
| 390 | own tenancy tests, which is not part of this spec. |
| 391 | |
| 392 | They follow `hibernation_configured`'s precedent: not `omitempty`, so absent |
| 393 | means "unknown" and false means "no". This is the one box-side change in this |
| 394 | spec. It is purely additive, and old readers ignore it. |
| 395 | |
| 396 | A code is **eligible** when at least one reachable box publishing it has |
| 397 | `hibernation_configured`, `serves_shared_pool` and `accepts_inbound` all |
| 398 | **positively true**. Unknown is not yes here. That is the reverse of |
| 399 | `placements`' generosity, and deliberately so: that generosity is right for |
| 400 | an explicit request, which a person made and can see refused. It is wrong for |
| 401 | an automatic one. |
| 402 | |
| 403 | Until boxes roll with the fields, no coded region is eligible. The guess then |
| 404 | picks only the home, so the feature cannot outrun the box roll. |
| 405 | |
| 406 | **The secrets key (R9).** An exec instance's config, secrets and store |
| 407 | binding reach it only through `<host>.config.env`, which the agent renders |
| 408 | only when the box holds `CARLOS_SECRETS_KEY` (`agent.go:881-886`, `:1812`). |
| 409 | A wake treats a missing file as empty (`backend_exec.go:1146-1149`), so a box |
| 410 | without the key runs a placed instance with nothing, silently. Eligibility |
| 411 | therefore gains a fourth fact. |
| 412 | |
| 413 | - **What a box publishes.** One more edges-file field, `secrets_key_fp`: the |
| 414 | fingerprint of the public half of the box's `CARLOS_SECRETS_KEY`, never the |
| 415 | key. The name, its place beside `placement` and `data_bucket`, and its |
| 416 | shape (additive, `omitempty`, absent when the box holds no key or a key |
| 417 | that does not parse) are the move-any-app spec's |
| 418 | (`2026-10-02-move-any-app-design.md` §6, on `design-moves-any-region` |
| 419 | at e41c4fef). Both features read one field. |
| 420 | - **The format**, which that spec names but does not fix, is fixed here for |
| 421 | both: `"sha256:"` followed by the lowercase hex SHA-256 of the public key's |
| 422 | uncompressed SEC1 bytes (`(*ecdh.PublicKey).Bytes()`, 65 bytes for P-256). |
| 423 | Those are exactly the bytes `CARLOS_SECRETS_PUBKEY` base64-decodes to, so a |
| 424 | box (from its private key) and the console (from the public key) compute the |
| 425 | same string. One helper, `appconfig.KeyFingerprint`, is the only code that |
| 426 | renders it. The move-any-app spec should cite it rather than define a |
| 427 | second one. |
| 428 | - **What the console expects.** The console learns the deployment's |
| 429 | fingerprint from its own `CARLOS_SECRETS_PUBKEY` (`Config.SecretsPub`, |
| 430 | `cmd/console/main.go:467-490`). That is the key the CLI seals config to, |
| 431 | so it is by definition the one a box must be able to open. The move-any-app |
| 432 | spec's console makes the same choice. The console never learns the |
| 433 | fingerprint from a box: a "majority of boxes" or "the home box" rule would |
| 434 | certify whatever key happened to be widespread, including a wrong one. |
| 435 | With `CARLOS_SECRETS_PUBKEY` unset, the console cannot judge. It logs that |
| 436 | once at startup, and no coded region is eligible for automatic placement. |
| 437 | - **The rule.** A code is eligible only if at least one box that meets the |
| 438 | other three facts also publishes a `secrets_key_fp` equal to the |
| 439 | console's. The home stays exempt, as it is from every eligibility fact: |
| 440 | it is where every legacy create already lands, and on the flagship it is |
| 441 | the one box that holds the key. |
| 442 | - **The reason word.** A code that fails only this fact reports |
| 443 | `no-secrets-key`, between `not-hosting` and `not-eligible` in the order of |
| 444 | checks (§9.3, C16). |
| 445 | |
| 446 | Candidate rules: |
| 447 | |
| 448 | - **Coded regions**: boxes whose `Placement` passes `provision.ValidRegion`. |
| 449 | Fleet codes (`fleet/<name>`) fail on the `/`. |
| 450 | - **The home**: boxes that publish no placement, or the home's name, and have |
| 451 | a dial address. The home is exempt from the positive check, because it is |
| 452 | where every legacy create already lands. `handleAPIRegions` already treats |
| 453 | it as always armed (`apiregions.go:42-47`). |
| 454 | - **AWS region**: the `Region` of the eligible boxes behind the code. If they |
| 455 | disagree, the most common wins, ties go to the lexically smallest, and the |
| 456 | disagreement is logged once. |
| 457 | - **Policy**, read with `provision.GetManifest` directly (codex 6). This is |
| 458 | never `instanceManifest`, which answers 409 or 502 for the caller |
| 459 | (`apiinstances.go:117`). |
| 460 | - *absent*: no allow-list and no pool. |
| 461 | - *known*: candidates pass through `instanceRegionAllowed` |
| 462 | (`apiinstances.go:1797`) on the **normalized** code. **A pooled manifest |
| 463 | yields no candidates and no guess** (codex 1). |
| 464 | - *unreadable*: no guess, and an explicit error at the guess endpoint. |
| 465 | - **Fleet placement** (codex R4-1). The instance guess endpoint (§9.4) |
| 466 | resolves `appPlacement` (`internal/console/fleets.go:479`) first. **A |
| 467 | fleet-placed app gets `"guess": null`**, because the door replaces any |
| 468 | submitted region with the fleet (`apiinstances.go:798`), so a guess could |
| 469 | only mislead a picker. A failed placement read is an explicit error there |
| 470 | (C27), never an unrestricted guess. App-creation inference (§5.5) is |
| 471 | unaffected: it chooses the stored app region, which stays dormant while the |
| 472 | app is placed. |
| 473 | - **Read failure** (codex R2-2): the listing failing is distinct from "no |
| 474 | file publishes a placement". `readEdgesFiles` (`instancesplacement.go:83`) |
| 475 | swallows the first today. A new `readEdgesFilesStrict` returns the error. |
| 476 | Candidates built after a failed read are the home alone, if the policy |
| 477 | admits it (§6.4), and the door treats the failure as unconfirmed (§7). |
| 478 | |
| 479 | ### 6.2 Points |
| 480 | |
| 481 | A point is a latitude and longitude held in memory for one request. |
| 482 | |
| 483 | **Timezone.** *(decision)* A new package, `internal/geo`, embeds three files |
| 484 | from one IANA tzdata release: |
| 485 | |
| 486 | - `zone.tab`, one row per country and zone, with ISO 6709 coordinates; |
| 487 | - a link table generated from tzdata's `Link` lines; |
| 488 | - a `VERSION` stamp. |
| 489 | |
| 490 | `zone.tab` is used rather than `zone1970.tab` because browsers report names |
| 491 | that `zone1970.tab` has merged away. For example, `Europe/Amsterdam` has its |
| 492 | own row in `zone.tab` and none in `zone1970.tab`. The links matter because |
| 493 | Chrome reports `Asia/Calcutta` where Firefox reports `Asia/Kolkata`. Go's |
| 494 | `time/tzdata` carries no coordinates, so it cannot be the source. |
| 495 | |
| 496 | The files are **kept current** by `scripts/update-tzgeo.sh <version>`, which |
| 497 | fetches the release from `data.iana.org`, checks its signature, and rewrites |
| 498 | the three files. Running it is a release-checklist step. A stale table only |
| 499 | misses zone names newer than it, and those fall through to IP. |
| 500 | |
| 501 | Lookup: |
| 502 | |
| 503 | 1. A shape check: at most 64 bytes, one to three `/`-separated segments of |
| 504 | `[A-Za-z0-9_+-]`. |
| 505 | 2. Unmappable by rule: `""`, `Local`, `UTC`, `UCT`, `GMT`, `Universal`, |
| 506 | `Zulu`, `Factory` and all of `Etc/`. |
| 507 | 3. Links, at most 4 hops. |
| 508 | 4. `zone.tab`, case-insensitive. |
| 509 | |
| 510 | A miss at any step makes the timezone unmappable. |
| 511 | |
| 512 | **IP.** *(decision)* `CONSOLE_GEOIP_DB` names an operator-supplied `.mmdb`. |
| 513 | It is read in `cmd/console/main.go` beside `CONSOLE_HOME_REGION` |
| 514 | (`main.go:278`) into a new `Config.GeoIP`. |
| 515 | |
| 516 | - **Recommended source**: DB-IP Lite (CC BY 4.0). Its attribution appears on |
| 517 | the page that uses it (C24) and in the README. MaxMind GeoLite2 also works, |
| 518 | under its EULA's conditions, which include attribution and keeping the data |
| 519 | updated. The operator owns that compliance. |
| 520 | - **Unset**: the IP fallback is off, which is the default. |
| 521 | - **Unopenable**: logged once, the fallback is off, and **the console still |
| 522 | starts**. |
| 523 | - **Lifecycle**: the file is opened once and memory-mapped. Replacing it is an |
| 524 | atomic rename plus a console restart. |
| 525 | - **Lookup**: the `location` field, else `country.iso_code`, else |
| 526 | `registered_country.iso_code`, through country points. A lookup error is a |
| 527 | miss, logged at most once a minute. |
| 528 | - **Never looked up**: loopback, private, link-local, CGNAT and unspecified |
| 529 | addresses. |
| 530 | |
| 531 | The reader is `github.com/oschwald/maxminddb-golang` (ISC), pinned to a |
| 532 | reviewed version. The tests use pinned fixtures from MaxMind's `MaxMind-DB` |
| 533 | test data (MIT or Apache-2.0). |
| 534 | |
| 535 | **Country points.** *(decision)* These come from the same `zone.tab`, not a |
| 536 | second dataset. |
| 537 | |
| 538 | - A one-row country uses its row. |
| 539 | - A multi-row country uses a shipped override naming its principal zone: |
| 540 | `US` → `America/New_York`, `AU` → `Australia/Sydney`, `CA` → |
| 541 | `America/Toronto`, `RU` → `Europe/Moscow`, `BR` → `America/Sao_Paulo`, |
| 542 | `CN` → `Asia/Shanghai`, and so on. |
| 543 | - A test fails when a tzdata refresh gives a country a second row with no |
| 544 | override. |
| 545 | |
| 546 | **Whose IP.** The client IP from `s.clientIP` reaches the guess only for a |
| 547 | person's request (§5.5). A service or app principal is a server, so it gets |
| 548 | the IP step only by sending an address in the guess body (§9.4). |
| 549 | |
| 550 | ### 6.3 Nearest |
| 551 | |
| 552 | Each eligible candidate's AWS region maps to a point through the region table |
| 553 | (§14). Distance is haversine (mean Earth radius 6371.0088 km), rounded to |
| 554 | whole kilometres. |
| 555 | |
| 556 | - A candidate with no point is left out and logged once per code. |
| 557 | - **If the home has no point, the guess is `default`.** |
| 558 | - **Ties** go to the home, then to the lexically smallest code. |
| 559 | - An ineligible candidate is never picked. |
| 560 | |
| 561 | ### 6.4 Order |
| 562 | |
| 563 | ``` |
| 564 | explicit region (validated as in §8.1) → "chosen" |
| 565 | else timezone gives a point → nearest → "timezone" |
| 566 | else IP gives a point → nearest → "ip" |
| 567 | else home → "default" |
| 568 | ``` |
| 569 | |
| 570 | **The guess invariant** (codex R3-4): every non-null guess, including every |
| 571 | fallback to the home, passes the same normalized policy predicate the door |
| 572 | applies (`instanceRegionAllowed` on the `homeRegionIn` form). The steps run in |
| 573 | this order: |
| 574 | |
| 575 | 1. Filter candidates by policy (§6.1). |
| 576 | 2. Pick the nearest among the filtered candidates that have a point. |
| 577 | 3. If the home has no point, or nothing maps, the fallback is the home, **but |
| 578 | only if the home is itself a filtered candidate**. |
| 579 | 4. Otherwise the answer is `"guess": null`. |
| 580 | |
| 581 | So a manifest that allows only `uk`, with the home's coordinates missing, |
| 582 | gives `uk` when the hint maps and `null` when it does not, never the home. |
| 583 | There is also no guess when the policy is pooled or unreadable. |
| 584 | |
| 585 | ### 6.5 Logged, and not logged (codex 8) |
| 586 | |
| 587 | - One line per app creation: |
| 588 | `console: app region <sqid>/<app>: <code> (<source>)`. |
| 589 | - Once per process: an unusable `CONSOLE_GEOIP_DB`; a code with no point; |
| 590 | boxes disagreeing on a code's AWS region. |
| 591 | |
| 592 | **The timezone, the IP and any coordinate never reach a URL, a log line, an |
| 593 | audit row, a response body or the database.** They travel only in POST |
| 594 | bodies. The canonical-host redirect copies the whole query string into its |
| 595 | `Location` header and its HTML (`server.go:1306`, `:1326`). A 308 on a POST |
| 596 | keeps the body out of both. |
| 597 | |
| 598 | ## 7. The instance create door |
| 599 | |
| 600 | **(decision) The fill, replacing `apiinstances.go:772`:** |
| 601 | |
| 602 | ```go |
| 603 | raw := strings.TrimSpace(req.Region) |
| 604 | regionFrom := "" |
| 605 | if raw != "" { |
| 606 | regionFrom = "request" |
| 607 | } |
| 608 | if raw == "" && placed == "" && m.Pool == "" && |
| 609 | strings.TrimSpace(req.Role) == "" && |
| 610 | (provision.Record{Kind: strings.TrimSpace(req.Kind), Addrs: req.Addrs}).DerivesPaths() { |
| 611 | appRegion, _, err := s.store.AppRegion(acct.ID, app) |
| 612 | if err != nil { |
| 613 | // log; 502 C20 |
| 614 | } |
| 615 | if appRegion != "" { |
| 616 | raw, regionFrom = appRegion, "app" |
| 617 | } |
| 618 | } |
| 619 | region := s.homeRegionIn(raw) |
| 620 | ``` |
| 621 | |
| 622 | **Where it sits.** After `appPlacement` (`apiinstances.go:763`), and before |
| 623 | `checkReplicaCreate` (`:781`), the allow-list (`:799`) and the armed gate |
| 624 | (`:827`). |
| 625 | |
| 626 | **Every check an explicit region meets, a defaulted one meets too** (R2). An |
| 627 | explicit home name is non-empty, so it never triggers the fill, and |
| 628 | `homeRegionIn` then turns it into `""`. That is the override that always |
| 629 | exists once §5.6 holds (codex 3). |
| 630 | |
| 631 | Each condition earns its place: |
| 632 | |
| 633 | | condition | why the default stays out | |
| 634 | | --- | --- | |
| 635 | | request names a region | an explicit choice wins (R2) | |
| 636 | | `placed != ""` | a fleet placement already governs every instance's region (`:785`) | |
| 637 | | `m.Pool != ""` | a defaulted region with no box serving the pool mints nowhere | |
| 638 | | `req.Role != ""` | a replica must name its placement; filling it would turn "role replica needs a placement" (`:578-581`) into a silent acceptance | |
| 639 | | not `DerivesPaths()` | only a socket-backed instance has a process and data that region places | |
| 640 | | store read fails | refused, as the placement read above it is (`:764-768`) | |
| 641 | |
| 642 | **An app-derived region needs positive eligibility** (codex R2-2). The armed |
| 643 | gate goes dormant when no file publishes a placement (`:848`). That |
| 644 | dormancy stays for explicit requests and for the legacy home default. For |
| 645 | `regionFrom == "app"` it does not apply. The door reads the edges files |
| 646 | strictly (§6.1) and refuses in three cases: |
| 647 | |
| 648 | - **The read failed**: a 503 with C28. The region could not be confirmed, |
| 649 | which is different from "absent". |
| 650 | - **No eligible box publishes the code**: C16, naming `absent`, `not-hosting`, |
| 651 | `no-secrets-key` (R9) or `not-eligible`. This covers the last box in a region being retired, and |
| 652 | every box losing its placement code. |
| 653 | - **The allow-list excludes it**: C17. |
| 654 | |
| 655 | The armed gate's own checks then run as before. |
| 656 | |
| 657 | *(decision)* **A failing defaulted region is refused, not quietly replaced by |
| 658 | the home.** The way out is always an explicit region, the home's name |
| 659 | included, and `app_default` (§9.3) and the Instances tab (§10) show the |
| 660 | refusal before anyone tries (codex 7). |
| 661 | |
| 662 | *(decision)* **A defaulted home whose region is in its own `replicas` set is |
| 663 | refused** (C26). Explicit requests keep today's behaviour. |
| 664 | |
| 665 | *(decision, R9)* **An explicit request into a region whose boxes provably |
| 666 | lack the deployment's secrets key is refused** (C31, 400), for a |
| 667 | socket-backed instance (`DerivesPaths`) and for a replica. Static and |
| 668 | addr-backed records need no config on the box and are unaffected. |
| 669 | |
| 670 | - **What "provably" means.** The console's `CARLOS_SECRETS_PUBKEY` is set, |
| 671 | and every reachable, hosting box publishing the code is a **current |
| 672 | writer** (it publishes `accepts_inbound`, which the same box release adds |
| 673 | and which is never omitted). Among those boxes, none publishes a matching |
| 674 | `secrets_key_fp`. A current writer that publishes no fingerprint holds no |
| 675 | key, because the field is omitted exactly then. |
| 676 | - **Unknown keeps today's behaviour.** An old writer, or a console with no |
| 677 | `CARLOS_SECRETS_PUBKEY`, cannot prove a mismatch, so the request proceeds |
| 678 | as it would today. Roll order therefore never refuses a working region. |
| 679 | - **Why refuse, and not keep today's behaviour.** The record this would |
| 680 | write is an instance that starts, answers health checks, and runs with no |
| 681 | config, no secrets and no store, which is the kpmg outage. It belongs with |
| 682 | the existing "reachable but cannot host" refusal (`apiinstances.go:827-852`, |
| 683 | #110). That one also refuses an explicit request the box would accept but |
| 684 | could never serve. A person who named the region learns why at the door, |
| 685 | not from a broken app. |
| 686 | |
| 687 | **Response.** `apiRecordView` (`apiinstances.go:535`) gains two fields that |
| 688 | sit outside the record, like `PrivateStoreBucket`: |
| 689 | |
| 690 | - `region_from`: `request`, `app` or `fleet`, omitted for the home default; |
| 691 | - `region_name` (§14). |
| 692 | |
| 693 | ## 8. Every create path |
| 694 | |
| 695 | ### 8.1 Where an app is created |
| 696 | |
| 697 | | path | file:line | change | |
| 698 | | --- | --- | --- | |
| 699 | | New-app form, GET | `handleNewAppForm`, `server.go:3586`; `app_new.html` | picker with an `@auto` option (§10), when the switch is on and two or more regions are eligible | |
| 700 | | New-app form, POST | `handleNewApp`, `server.go:3607`; claim at `:3626` | `region` is `@auto` or a code; `tz` from a hidden field; `ip` from `s.clientIP`; builds `regionReq` (§5.5) | |
| 701 | | CLI API create | `handleAPICreateApp`, `internal/console/apicli.go:330`; body at `:359`; claim at `:415`; response at `:461` | takes `region` and `timezone`; `ip` from `s.clientIP` for a person; returns `region`, `region_name`, `region_source` and `regions_available` | |
| 702 | | The claim | `claimApp`, `server.go:3466`; insert at `:3550` | takes `regionReq`; resolves once it knows whether this is an adoption; `CreateAppInRegion` (§5.3) | |
| 703 | | `carlos apps create` | `runAppsCreate`, `apps.go:442`; call at `:497`; print at `:502` | `--region`; sends the timezone; prints the region line (§11) | |
| 704 | | `carlos deploy` onboarding | `offerCreateApp`, `deploy.go:513`, reached from `runDeploy` at `:113`; call at `:536`; print at `:540` | `--region`; sends the timezone; prints the region line | |
| 705 | | CLI client | `apiSession.createApp`, `apimode.go:994`; `apiCreateAppResult`, `:958` | two `omitempty` fields in, four tolerant fields out | |
| 706 | | Legacy owners backfill | `BackfillOpsMemberships`, `store.go:623`; insert at `:665` | none: the defaults mean the home | |
| 707 | |
| 708 | *(decision)* An explicit region at app create is validated the way the |
| 709 | instance door validates one: |
| 710 | |
| 711 | 1. `ValidRegion`. |
| 712 | 2. Armed and hosting, unless the gate is dormant. |
| 713 | 3. Not provably missing the secrets key (R9, the same rule as §7), refused |
| 714 | with C31. |
| 715 | 4. The home's name is accepted through `homeRegionIn`. |
| 716 | |
| 717 | Refusals list what would pass (C18, C19, C31). There is no allow-list yet, because |
| 718 | there is no manifest yet. A fleet `place` and a `region` may both be sent, |
| 719 | and both are stored (§12). |
| 720 | |
| 721 | ### 8.2 Where an instance is created |
| 722 | |
| 723 | | path | file:line | today | change | |
| 724 | | --- | --- | --- | --- | |
| 725 | | The door | `handleAPIInstanceCreate`, `apiinstances.go:608`; region at `:772` | `""` means the home | the fill and eligibility (§7) | |
| 726 | | `carlos instances create` | `instances_cli.go:322` | sends `--region` or `""` | help text; prints `region_name` (§11) | |
| 727 | | `carlos canary` | `canary.go:407` | sends `""` | the door fills the app's region; a new `--region`, with the existing-canary rule (§11) | |
| 728 | | `carlos deploy` static host | `mintPlatformHost`, `deploy.go:858` | static, `""` | none | |
| 729 | | First-route domain attach | `internal/console/domainattach.go:178` | static, no region | none | |
| 730 | | External callers (titogo, jam) | `POST …/instances`, `internal/console/authcli.go:613` | usually `""` | follow the app's region; can ask for a guess (§9.4) | |
| 731 | | Sidecar create | `handleAPISidecarCreate`, `internal/console/apisidecars.go:257` | `""` or explicit | none (§12) | |
| 732 | |
| 733 | ## 9. API changes |
| 734 | |
| 735 | All changes are additive: new request fields are optional and new response |
| 736 | fields can be ignored. `readJSON` is not strict (`authcli.go:932-947`), so an |
| 737 | old console ignores the new request fields instead of refusing them. |
| 738 | |
| 739 | **9.1 `POST /api/cli/apps/{account}/new`.** The request gains two optional |
| 740 | fields: |
| 741 | |
| 742 | - `region`: an explicit choice, as a code or the home's name; |
| 743 | - `timezone`: a hint, used only when there is no `region`, and never stored. |
| 744 | |
| 745 | The response gains these fields, echoing what was **stored**: |
| 746 | |
| 747 | ```json |
| 748 | {"region": "uk", "region_name": "UK · London", "region_source": "timezone", |
| 749 | "regions_available": 3} |
| 750 | ``` |
| 751 | |
| 752 | - `regions_available` counts eligible candidates. |
| 753 | - With the switch off, an explicit non-home `region` is answered with the |
| 754 | home and `default`, and the CLI says so (C13b). |
| 755 | |
| 756 | **9.2 `POST /api/cli/apps/{account}/{app}/instances`.** The request is |
| 757 | unchanged and behaviour follows §7. The response gains `region_from` and |
| 758 | `region_name`. |
| 759 | |
| 760 | **9.3 `GET /api/cli/apps/{account}/{app}/regions`.** The gate, the role |
| 761 | (`authcli.go:586`) and the legacy fields are unchanged (codex 6): |
| 762 | |
| 763 | - `regions` is still the observed topology, unfiltered. |
| 764 | - `default` is still `HomeRegion`. |
| 765 | - Each entry gains `aws_region` and `name`. |
| 766 | |
| 767 | The response gains one block: |
| 768 | |
| 769 | ```json |
| 770 | "app_default": { |
| 771 | "region": "uk", "name": "UK · London", "source": "timezone", |
| 772 | "effective": "uk", |
| 773 | "available": false, "reason": "absent", |
| 774 | "region_known": true, "placement_known": true, |
| 775 | "instance_domain": "bab.oncarlos.com" |
| 776 | } |
| 777 | ``` |
| 778 | |
| 779 | - `region` is the app's **stored** region under its name, always. It is the |
| 780 | only field `deploy --region` compares against (codex R2-4). |
| 781 | - `effective` is where a create that names no region actually lands: |
| 782 | - the stored region; |
| 783 | - the home, for an app with no region of its own or a pooled app; |
| 784 | - the fleet placement itself (`fleet/acme`), for a fleet-placed app, never |
| 785 | the platform home (codex R2-4). |
| 786 | - `available` and `reason` describe `effective`: |
| 787 | - `absent`: the code is reported even though `regions` does not list it |
| 788 | (codex 7); |
| 789 | - `not-hosting`; |
| 790 | - `no-secrets-key` (R9, §6.1); |
| 791 | - `not-eligible` (§6.1); |
| 792 | - `not-allowed`; |
| 793 | - `pooled`; |
| 794 | - `fleet`: `available` is omitted, because the fleet's own rail decides; |
| 795 | - `policy-unreadable`: `available` is omitted; |
| 796 | - `edges-unreadable`: `available` is omitted. |
| 797 | - An absent manifest is not an error, and no query parameters are added |
| 798 | (codex 8). |
| 799 | - **(R9a) Two knowledge flags.** `/regions` always answers 200, so these |
| 800 | say which facts behind `app_default` the console actually read: |
| 801 | - `region_known` is true exactly when the app's stored region was read. |
| 802 | When it is false, `region`, `name`, `source` and `effective` are empty |
| 803 | (never the home's name standing in for an unread value), and |
| 804 | `available` is omitted. |
| 805 | - `placement_known` is true exactly when the app's manifest and its fleet |
| 806 | placement were read. When it is false, `reason` is `policy-unreadable` |
| 807 | and `available` is omitted. |
| 808 | - They separate certainty about the stored region from certainty about |
| 809 | the placement policy, because the two CLI checks need different facts. |
| 810 | - The R9a emptiness holds whatever the placement says: for a pooled or |
| 811 | fleet-placed app whose stored region was not read, `effective` is still |
| 812 | empty and `available` still omitted, and `reason` (`pooled`, `fleet`) |
| 813 | is the only placement fact stated (rev 6.2, Astra plan round 4, |
| 814 | finding 1). |
| 815 | - **Absent knowledge fails closed.** A client treats a flag that is false, |
| 816 | or missing from an `app_default` block, as unknown, and refuses the |
| 817 | check that needs it. `deploy --region` needs `region_known`; refused, |
| 818 | it prints C15c. `canary --region` needs `placement_known` and, because |
| 819 | its fleet check reads `effective`, `region_known` too (rev 6.2); |
| 820 | refused, it prints C30b. |
| 821 | - Both flags are additive. A console too old to send `app_default` at |
| 822 | all is handled as before (§9.6, §11): deploy skips its check, and |
| 823 | canary refuses `--region`. |
| 824 | - **(rev 6.2) `instance_domain`.** Additive and read-only: the domain the |
| 825 | app's instance hosts take, so a client can name an instance's host before |
| 826 | writing anything. It is the manifest's `domain` when a manifest exists, |
| 827 | else the domain the instances-manifest PUT would stamp |
| 828 | (`<sqid>.<PLATFORM_APPS_DOMAIN>`), and it is omitted when the manifest is |
| 829 | unreadable or the console has no valid apps domain. |
| 830 | - It is the authoritative hostname suffix for `canary --region`'s |
| 831 | preflight: the CLI builds `<name>.<app>.<instance_domain>` and reads that |
| 832 | one record (`GET …/instances/{host}`) before any write, because the |
| 833 | capped instance list skips records it cannot read. A 404 means no |
| 834 | canary yet; any other failure, or no `instance_domain`, refuses with |
| 835 | C30c. |
| 836 | - The CLI then checks the instances-manifest PUT's answer against the |
| 837 | preflight: the host built from the domain that PUT returns must equal |
| 838 | the host it read. If it does not, it stops with C30c before writing any |
| 839 | record. |
| 840 | |
| 841 | **9.4 `POST /api/cli/apps/{account}/{app}/regions/guess`.** |
| 842 | |
| 843 | - Same gate and role as §9.3. |
| 844 | - The body is `{"tz": "...", "ip": "..."}`, both optional. A person's request |
| 845 | with no `ip` uses its own address, and a service principal's never does. |
| 846 | - The answer is `{"guess": {"code", "name", "source"}}` or `{"guess": null}`. |
| 847 | It is `null` for a pooled or fleet-placed app (§6.1) (codex R4-1). |
| 848 | - An unreadable manifest, or an unreadable fleet placement, gets a 502 with |
| 849 | C27. |
| 850 | - It writes nothing. |
| 851 | |
| 852 | **9.5 Browser: `POST /a/{sqid}/apps/new/region`.** |
| 853 | |
| 854 | - A form body with `csrf` and `tz`. |
| 855 | - Gated by session through `s.account`. |
| 856 | - Returns `{"code", "name", "source"}`. |
| 857 | - Writes nothing. |
| 858 | - `new` is a reserved app name (`server.go:1150`). |
| 859 | |
| 860 | **9.6 Compatibility and mixed versions.** |
| 861 | |
| 862 | | client | console | result | |
| 863 | | --- | --- | --- | |
| 864 | | old CLI | new | No `timezone` is sent, so IP for a person, then home. Never an error. | |
| 865 | | new CLI | old | The response has no `region`. The CLI prints nothing, or C13a if `--region` was given. | |
| 866 | | new CLI | new, switch off | The app goes to the home. With `--region`, C13b. | |
| 867 | | titogo, unchanged | new | Creates follow the app's region, which for every existing app is the home. `default` is unchanged. | |
| 868 | | any | rolled back below the floor | R7: no-region creates land at the home, and roll-forward lists the possible discrepancies (§5.7). | |
| 869 | |
| 870 | ## 10. Console UI |
| 871 | |
| 872 | **New-app page** (`app_new.html`), when the switch is on and two or more |
| 873 | regions are eligible: |
| 874 | |
| 875 | - **The picker.** A `<select name="region">` whose first option has the value |
| 876 | `@auto` and the label C3, naming the server's guess (IP, then home). Every |
| 877 | eligible region follows, with its code as the value and its place name as |
| 878 | the label (C1, C2, C23). A hidden `tz` field rides along. |
| 879 | - **`@auto` is not a region** (codex 2, codex R2-3). Selecting a code is the |
| 880 | person's choice, and leaving `@auto` asks the server to guess. |
| 881 | - **The script.** An inline progressive-enhancement script, in the style of |
| 882 | the account switcher (`templates/layout.html:583`), fills `tz`, POSTs it to |
| 883 | §9.5, and relabels only the `@auto` option. |
| 884 | - **Otherwise** there is no picker, and the form is byte-identical to today's. |
| 885 | - **Attribution.** DB-IP attribution (C24) shows when the IP step is |
| 886 | configured. |
| 887 | |
| 888 | **POST** (`handleNewApp`): |
| 889 | |
| 890 | - An eligible code is `explicit`. |
| 891 | - `@auto`, or no `region` field at all, makes the server resolve from `tz`, |
| 892 | then the request's IP, then the home. On an adoption that resolves to `""`. |
| 893 | - Any other value re-renders the form with C5. |
| 894 | - The success flash (`server.go:3644`) is existing copy and is untouched. The |
| 895 | audit detail (`:3643`) gains the place (C21). |
| 896 | |
| 897 | **Instances tab** (the Placement section, `app_instances.html:60-78`), from |
| 898 | `app_default`: |
| 899 | |
| 900 | - C8 shows whenever the default cannot accept a create. |
| 901 | - Otherwise C7 shows when the app has a non-home region, or when two or more |
| 902 | regions are eligible. |
| 903 | - Pooled and fleet-placed apps show neither line. |
| 904 | |
| 905 | ## 11. CLI |
| 906 | |
| 907 | **Timezone detection** is a helper in the root package, with `getenv`, |
| 908 | `readlink` and `readFile` injected and only portable calls, so `20-cross` |
| 909 | builds it everywhere. It tries, in order: |
| 910 | |
| 911 | 1. `$TZ`, with a leading `:` stripped; a path keeps what follows `zoneinfo/`. |
| 912 | 2. `readlink /etc/localtime`, keeping what follows `zoneinfo/`. |
| 913 | 3. The first line of `/etc/timezone`. |
| 914 | 4. Nothing. |
| 915 | |
| 916 | On Windows only a valid `$TZ` is sent. Otherwise the console falls back to |
| 917 | IP, then the home. |
| 918 | |
| 919 | | command | flag | meaning | |
| 920 | | --- | --- | --- | |
| 921 | | `carlos apps create` | `--region <code>` | The app's region. The default is resolved by the console. | |
| 922 | | `carlos deploy` | `--region <code>` | The region of the app this deploy creates. On an existing app, it compares against `app_default.region` (the stored region) (codex R2-4): equal is silent, and **different is refused before anything ships** (C15). It is skipped against a console with no `app_default`. A fleet-placed app is judged by its stored region too, because the flag names a region, not a fleet. | |
| 923 | | `carlos canary` | `--region <code>` | The canary's region. The default is the app's. Before either branch, the CLI reads `app_default.effective` (§9.3). **If the app is fleet-placed, `--region` is refused before anything is created, rechannelled or uploaded, on the first run and on every repeat** (C30) (codex R4-2): the fleet governs every instance's region, so the flag could never be honoured, and refusing it the same way both times keeps the command's answer independent of whether the canary exists yet. Otherwise, for an existing canary host, the record's region is compared with the flag, reading `""` as the home's name: equal proceeds, and **different is refused before any record write, rechannel or upload** (C25) (codex 5). Without `--region`, a fleet-placed app's canary is created and reused exactly as today. | |
| 924 | | `carlos instances create` | `--region <code>` | Unchanged. The help says the default is now the app's region. | |
| 925 | |
| 926 | Output: |
| 927 | |
| 928 | - After "created app …" (`apps.go:502`, `deploy.go:540`), one line per |
| 929 | source, C9 to C12, printed only when `regions_available >= 2`. |
| 930 | - `instances create` and `canary` add the place to their existing line (C14). |
| 931 | - No prompts (R4). |
| 932 | |
| 933 | `docs/site/cli.md` gains the new flags and the new default. There is no new |
| 934 | command. |
| 935 | |
| 936 | ## 12. Interactions |
| 937 | |
| 938 | - **Customer fleets.** `appPlacement` (`internal/console/fleets.go:479`) |
| 939 | overrides the request's region (`apiinstances.go:785-797`). The precedence |
| 940 | is explicit: **fleet placement, then the request's region, then the app's |
| 941 | region, then the home.** A placed app's region is stored, stays dormant, |
| 942 | and is reported separately from its placement (§9.3). A placed app gets no |
| 943 | instance guess (§6.1), and `canary --region` refuses on it consistently |
| 944 | (§11) (codex R4-1, R4-2). |
| 945 | - **Pools** (`Manifest.Pool`). There is no default fill and no guess, and |
| 946 | `app_default` reports `pooled`. An explicit `--region` |
| 947 | behaves as today. |
| 948 | - **Pool-restricted and no-inbound boxes** (codex R2-1) are never eligible |
| 949 | for an automatic choice (§6.1). Explicit requests keep today's gate. |
| 950 | - **Replicas (#162).** A `role: replica` create is never filled. A |
| 951 | defaulted home inside its own `replicas` set is refused (§7). |
| 952 | - **Moves (#175/#331).** A move rewrites one record's `region` at its cut. |
| 953 | **It never touches the app's region**. Steering for moved instances is the |
| 954 | follow-on spec's, derived from region (§13.1). |
| 955 | - **Static.** Not filled. Its existing latency default is subject to the |
| 956 | §13.3 gate like everything else steered. |
| 957 | - **Private storage (#412).** Unaffected. |
| 958 | - **Sidecars.** Unchanged. Their door has no armed gate, so following the app |
| 959 | region needs its own design (§20). |
| 960 | - **Allow-lists and the home's name.** `instanceRegionAllowed` compares the |
| 961 | normalized `""`, so a manifest listing the home's code refuses creates at |
| 962 | the home today. This change keeps that and reinterprets no manifest. The |
| 963 | README documents it, and rollout audits production manifests first (§17). |
| 964 | |
| 965 | ## 13. Latency routing (R6): the requirements Phase L must meet |
| 966 | |
| 967 | ### 13.1 Why this is now a separate spec (codex R3-1, R3-2, R3-5) |
| 968 | |
| 969 | Rev 3 treated latency routing as two small things: a stamp on each non-home |
| 970 | record at mint, and arming the existing converger (#169). Round 3 showed that |
| 971 | the converger itself is not ready to arm. Each of these three faults was |
| 972 | checked against the code: |
| 973 | |
| 974 | - **Arming can break DNS for names nobody steered** (codex R3-1). The flagship |
| 975 | answers every tenant name with one `*.oncarlos.com` record pointing at the |
| 976 | Ireland EIP (carloku-infrastructure `dns53.tf` and `dns.tf`; the platform |
| 977 | README documents the same single wildcard at `README.md:442`). The converger |
| 978 | writes A records at exact names (`internal/steering/steering.go:369`, `:386`). |
| 979 | One latency record at `new.bap.oncarlos.com` creates the empty non-terminal |
| 980 | `bap.oncarlos.com`. From then on, `old.bap.oncarlos.com` finds `bap` as its |
| 981 | closest encloser, looks for `*.bap.oncarlos.com`, which does not exist, and |
| 982 | no longer matches `*.oncarlos.com` (RFC 4592 §3.3.1). Every unsteered |
| 983 | sibling under that account stops resolving. Static platform hosts have been |
| 984 | stamped `latency` at mint for weeks (`apiinstances.go:936`), so **arming #169 |
| 985 | as the code stands would do this to every account with a static site.** It |
| 986 | would happen with or without this spec. |
| 987 | - **Two edges in one AWS region produce an invalid record set** (codex R3-2). |
| 988 | `desiredRRSet` keys each record by box (`SetIdentifier`) but stamps it with |
| 989 | the edge's AWS region (`internal/steering/route53.go:597`). Route53 allows |
| 990 | one latency record per region within a name and type. A second London box |
| 991 | makes every batch fail, on every pass. This spec allows several placement |
| 992 | codes in one AWS region (§6.1), so the topology is reachable. |
| 993 | - **Large withdrawals never complete** (codex R3-5). When a pass would delete |
| 994 | more than the shrink limit, it refuses **every** change and returns |
| 995 | (`route53.go:207`). The loop's own notes say this repeats forever until an |
| 996 | operator steps in (`internal/host/steering_loop.go:154`). Rev 3's claim that |
| 997 | deletes "spread over passes" was wrong. |
| 998 | |
| 999 | Behind these sits a fourth fact. The flagship's Route53 zone for |
| 1000 | `oncarlos.com` is marked "STAGED, NOT YET AUTHORITATIVE" in `dns53.tf`, and |
| 1001 | DNSimple still serves the live answers (`dns.tf`). A converger writing to a |
| 1002 | zone nobody consults is inert at best. |
| 1003 | |
| 1004 | The arming hazard was posted to amadan discussion carlos/platform #169 on |
| 1005 | 2026-10-01, so it is on record where the arming decision lives, independent |
| 1006 | of this spec. |
| 1007 | |
| 1008 | Three rounds have each found a new fault in this one area. Per the lesson this |
| 1009 | repo has already paid for, the answer is not another patch to an example. It |
| 1010 | is an invariant, a design built to it, and a simulation that checks it. |
| 1011 | That is a spec of its own, about the converger. It is not a section of a |
| 1012 | spec about choosing regions. |
| 1013 | |
| 1014 | **(ruling R8) Phase L moves to a follow-on spec, and placement ships without |
| 1015 | waiting for it.** R6 is unchanged: latency routing for non-home instances is |
| 1016 | still in scope for this work and still binding. It is sequenced, not dropped. |
| 1017 | This spec now does three things for R6: |
| 1018 | |
| 1019 | 1. It **writes no stamp.** Rev 3's mint-time `Steering: "latency"` is |
| 1020 | withdrawn. Every stamped record would join the set of names whose DNS |
| 1021 | changes when the zone is armed, and that set is exactly the danger above. |
| 1022 | The follow-on spec should derive "steer this host" from the record's |
| 1023 | region, not from a stamp: any socket-backed home record with a non-home |
| 1024 | region, in the shared pool and the platform namespace. That covers |
| 1025 | instances created before it lands, and instances moved out of the home, |
| 1026 | with no backfill and no change to the move code. |
| 1027 | 2. It **states the invariants** that spec must meet (§13.2). |
| 1028 | 3. It **gates arming #169** on that spec (§13.3). |
| 1029 | |
| 1030 | ### 13.2 Invariants for the follow-on spec |
| 1031 | |
| 1032 | The DNS model is the authoritative zone for the apps domain, the set of armed |
| 1033 | edges with their AWS regions and health, and the bucket records. Over that |
| 1034 | model, these must hold after every converge pass, armed or not: |
| 1035 | |
| 1036 | - **I1, continuity.** Arming, and every later pass, leaves the answer |
| 1037 | unchanged for every name that has no steered record set of its own. This |
| 1038 | includes names that sit beneath an empty non-terminal a steered name |
| 1039 | creates, nested canaries, names whose steering was turned off, and names |
| 1040 | whose record was deleted. "The answer" is the result of RFC 4592 |
| 1041 | resolution over the whole zone, not an exact-name lookup. |
| 1042 | - **I2, validity.** For every steered name, the zone holds at most one latency |
| 1043 | record per AWS region. The record set is a function of the set of eligible |
| 1044 | edges serving that name, **grouped by AWS region**. Adding, removing or |
| 1045 | renaming a box changes the answers inside a region's group. It never |
| 1046 | changes how many latency records the name has for that region. One way to |
| 1047 | get this is one latency record per region, aliasing a per-region, |
| 1048 | health-checked set of that region's edges. The spec chooses. |
| 1049 | |
| 1050 | This groups by region rather than "keyed by edge" because Route53's rule is |
| 1051 | per region. |
| 1052 | - **I3, serviceability.** Every answer for a steered name is an edge that can |
| 1053 | serve it: healthy, holding the account's certificate, and able to reach the |
| 1054 | name's home through the home map. The far edge backhauls; it never |
| 1055 | answers 404. |
| 1056 | - **I4, withdrawal.** A withdrawal justified by a complete read (a record |
| 1057 | deleted, steering turned off, a box retired) converges within a bounded |
| 1058 | number of passes. If it cannot, it raises a named operator alert with a |
| 1059 | documented procedure. The short-read guard still refuses deletions that a |
| 1060 | complete read does not justify, but a refusal of deletions never blocks |
| 1061 | unrelated additions in the same pass. |
| 1062 | - **I5, dormancy.** With no zone configured, nothing is read and nothing is |
| 1063 | written. This is already true (`steering_loop.go:219`). |
| 1064 | - **I6, authority.** The converger only writes to the zone the apps domain is |
| 1065 | actually delegated to. It verifies the delegation before its first write, |
| 1066 | and stays dormant, loudly, when it cannot. |
| 1067 | |
| 1068 | **The proof the follow-on spec must ship:** a property test over generated |
| 1069 | fleets. It covers random edges, random AWS regions including duplicates, |
| 1070 | random steered and unsteered names including nested and sibling names, and |
| 1071 | random sequences of creates, deletes, opt-outs, moves and box retirements. |
| 1072 | |
| 1073 | Each scenario runs through `desiredState` and `converge` against a Route53 |
| 1074 | fake. The fake enforces the per-region latency rule, batch atomicity, and an |
| 1075 | RFC 4592 resolver. The test asserts I1 to I5 after every step, at high seed |
| 1076 | counts, and each invariant's assertion is checked by mutation: break the |
| 1077 | code, and watch the test fail. Examples alone have now missed three faults |
| 1078 | in a row. |
| 1079 | |
| 1080 | ### 13.3 Gates |
| 1081 | |
| 1082 | - **Do not arm #169** (`CARLOS_STEERING_ZONE_ID`) on any deployment until the |
| 1083 | follow-on spec has landed and its simulation passes. This holds for the |
| 1084 | static-site steering that already exists, independently of this spec. |
| 1085 | - The follow-on spec carries the infrastructure plan for scoped approval |
| 1086 | (AGENTS.md): |
| 1087 | - the zone and its delegation; |
| 1088 | - `steering_zone_ids` and `steering_record_names = ["*.<apps-domain>"]` on |
| 1089 | every platform box's `region-host` module |
| 1090 | (`infra/modules/region-host/main.tf:281`, `variables.tf:153`); |
| 1091 | - `CARLOS_STEERING_ZONE_ID` in each box's `host.env`; |
| 1092 | - cost, cleanup and disarm. |
| 1093 | - It also carries the latency measurement. Rev 3's method stands and moves |
| 1094 | there: `dig +subnet` against the zone's nameservers, then |
| 1095 | `curl -w '%{time_connect} %{time_appconnect} %{time_starttransfer}'` from a |
| 1096 | vantage in each region, 30 runs per host, with medians and p90. Take a |
| 1097 | baseline after this spec's placement ships and before arming, measure again |
| 1098 | after, and record the numbers in #169. |
| 1099 | |
| 1100 | Until then, a deployment that runs this spec is **placement-only**. A UK |
| 1101 | instance is reached through the deployment's static DNS (on the flagship, the |
| 1102 | Ireland EIP) and backhauled to the UK box. That costs the visitor about one |
| 1103 | Ireland–London round trip compared with landing in London. It is no worse |
| 1104 | than today for anyone, and it relieves Ireland's disk for every new non-home |
| 1105 | instance. |
| 1106 | |
| 1107 | ## 14. Place names |
| 1108 | |
| 1109 | **(decision) One table, `internal/geo/awsregions.go`, maps an AWS region code |
| 1110 | to `{latitude, longitude, place}`.** |
| 1111 | |
| 1112 | - It covers the AWS commercial regions on AWS's official list as of |
| 1113 | 2026-10-01 (C23), and records that source and date in a comment. |
| 1114 | - Coordinates are the metro area's, each checked by hand when the table is |
| 1115 | built. |
| 1116 | - A region missing from the table shows its bare code and is left out of the |
| 1117 | guess, with a log line. |
| 1118 | - The table is used everywhere: the picker, the Instances tab, every `name` |
| 1119 | in §9, and the CLI lines. |
| 1120 | |
| 1121 | **`STATUS_BOX_REGION` stays, with a different job.** It is the public status |
| 1122 | page's commitment: operator free text, deliberately coarse, and "never derived |
| 1123 | from cloud metadata" (`agent.go:88-92`, README `:578`). It may stay coarser |
| 1124 | than the picker, but it must agree geographically, and the README says so. |
| 1125 | |
| 1126 | ## 15. Failure modes |
| 1127 | |
| 1128 | | what fails | what happens | |
| 1129 | | --- | --- | |
| 1130 | | `CONSOLE_GEOIP_DB` unset | IP step off: timezone, then home | |
| 1131 | | mmdb missing, unreadable or corrupt at startup | Logged once; IP step off; the console starts. | |
| 1132 | | mmdb lookup error | A miss; rate-limited log. | |
| 1133 | | timezone UTC, `Etc/*`, unknown or malformed | IP step, then home. | |
| 1134 | | boxes not yet publishing the eligibility fields | No coded region is eligible. The guess is the home, and app-derived defaults are refused as `not-eligible`. | |
| 1135 | | a region's boxes hold no secrets key, or a different one (R9) | Not eligible for any automatic choice. App-derived defaults are refused with C16 `no-secrets-key`. Explicit socket-backed or replica creates are refused with C31 once a current writer proves the mismatch. | |
| 1136 | | console has no `CARLOS_SECRETS_PUBKEY` (R9) | Logged once. No coded region is eligible for automatic placement. Explicit creates keep today's behaviour. | |
| 1137 | | edges listing fails | The guess is the home. App-derived creates get 503 C28. Explicit creates are unchanged. | |
| 1138 | | last box of the app's region retired | App-derived creates are refused (C16 `absent`), and `app_default` and C8 say why. | |
| 1139 | | home publishes no AWS region | Every guess is `default`. Fix it with the box's `CARLOS_AWS_REGION`. | |
| 1140 | | an eligible region's AWS region is not in the table | Left out of the guess, still selectable, shown by code. | |
| 1141 | | app's region excluded by a later allow-list | Refused (C17). | |
| 1142 | | app-region read fails at the door | 502 (C20); no record written. | |
| 1143 | | manifest unreadable | The guess endpoint returns 502 (C27). `/regions` answers `policy-unreadable`. | |
| 1144 | | switch on, home unnamed | The switch is treated as off; logged once. | |
| 1145 | | console binary rolled back below the floor | R7: no-region creates land at the home; roll-forward lists possible discrepancies. | |
| 1146 | | steering unarmed | Placement-only, with no error. A non-home instance is reached through the static DNS and backhauled (§13.3). | |
| 1147 | | a manifest's allow-list excludes the home, and no hint maps | `"guess": null`, never the home (§6.4). | |
| 1148 | | owner adopts an existing app | `""`/`default`, unless `region` is explicit. | |
| 1149 | |
| 1150 | ## 16. Test plan |
| 1151 | |
| 1152 | **`internal/geo`.** |
| 1153 | |
| 1154 | - Haversine on known pairs: Dublin to London is about 464 km, within 5. |
| 1155 | - Nearest: |
| 1156 | - a single candidate, and a clear winner; |
| 1157 | - ties go to the home, then the smaller code; |
| 1158 | - two codes in one AWS region; |
| 1159 | - an ineligible candidate nearer than every eligible one is never picked; |
| 1160 | - a candidate with no point is excluded, a home with no point gives |
| 1161 | `default`, and no candidates gives `default`. |
| 1162 | - Timezones: |
| 1163 | - `Europe/London` and `Europe/Amsterdam` map; |
| 1164 | - `Asia/Calcutta` and `Asia/Kolkata` give one point; |
| 1165 | - `America/Argentina/Buenos_Aires` maps, and matching is case-insensitive; |
| 1166 | - `UTC`, `Etc/UTC`, `Etc/GMT+5`, `GMT`, `Factory`, `Local`, `""`, a |
| 1167 | 65-byte name, `../etc/passwd` and a link loop are all unmappable. |
| 1168 | - Country points: every multi-row country has an override. |
| 1169 | - mmdb, against pinned fixtures: |
| 1170 | - unset, missing, truncated and non-mmdb files; |
| 1171 | - City and Country-only records; |
| 1172 | - private, loopback and CGNAT addresses never reach the reader. |
| 1173 | |
| 1174 | **Secrets key (R9):** |
| 1175 | |
| 1176 | - `appconfig.KeyFingerprint` of a key's public half equals the fingerprint |
| 1177 | computed from the base64 `CARLOS_SECRETS_PUBKEY` of the same key. |
| 1178 | - A box with the key publishes `secrets_key_fp`. With no key, or a key that |
| 1179 | does not parse, it omits the field. The body never contains the private key |
| 1180 | or the public key itself. |
| 1181 | - Eligibility: |
| 1182 | - a box with a matching fingerprint is eligible; |
| 1183 | - a box with a different fingerprint, or none, is `no-secrets-key`; |
| 1184 | - a console with no `CARLOS_SECRETS_PUBKEY` makes no coded region |
| 1185 | eligible. |
| 1186 | - An explicit socket-backed create into a region where only current writers |
| 1187 | publish a mismatch is refused with C31. Under an old writer, or with no |
| 1188 | console key, it is accepted as today. Static and addr-backed creates are |
| 1189 | never refused for it. |
| 1190 | |
| 1191 | **Edges file and eligibility** (codex R2-1): |
| 1192 | |
| 1193 | - The writer publishes `serves_shared_pool` and `accepts_inbound` from |
| 1194 | `CARLOS_POOLS` and `CARLOS_INBOUND`, for: |
| 1195 | - unset; |
| 1196 | - `CARLOS_POOLS=tito`; |
| 1197 | - `CARLOS_POOLS=,tito`, which the parser reads as `["tito"]`, so it |
| 1198 | publishes `serves_shared_pool: false` (codex R3-3); |
| 1199 | - `CARLOS_INBOUND=none`. |
| 1200 | - The published fields are asserted equal to `poolServed` and |
| 1201 | `cfg.Inbound != InboundNone` over the same `provisionTickConfig` output, for |
| 1202 | a table of `CARLOS_POOLS` and `CARLOS_INBOUND` spellings. That makes |
| 1203 | publication and reconciliation one predicate. |
| 1204 | - The reader treats absent as unknown and false as no. |
| 1205 | - Eligibility needs all three fields true on one reachable box. |
| 1206 | - The guess against a named-pool-only box returns no guess for that code. |
| 1207 | - The guess against a no-inbound box does the same. |
| 1208 | - In both cases, the matching reconcile pass on that box refuses the record |
| 1209 | (`reconcile.go:925`, `:1339`). That pins why the guess must not go there. |
| 1210 | |
| 1211 | **Store.** |
| 1212 | |
| 1213 | - Four database states: fresh; from before the columns; from before |
| 1214 | `migrateAppsKey` but carrying the new ALTERs; and `migrate` run twice. |
| 1215 | - Pre-existing rows read `""`/`""`. |
| 1216 | - `CreateAppInRegion` writes both columns in one statement. |
| 1217 | - Rename and transfer keep the region. |
| 1218 | |
| 1219 | **The door** (`apiinstances_test.go`): |
| 1220 | |
| 1221 | - No region, with app region `uk` eligible: the record says `uk` and |
| 1222 | `region_from` says `app`. |
| 1223 | - App region `""`: the record is byte-identical to the pre-change golden. |
| 1224 | - Explicit wins over the app, and fleet wins over both. |
| 1225 | - An explicit home name gives `""`. |
| 1226 | - Pooled, static, addr-backed and replica creates are not filled, and the |
| 1227 | replica refusal is word for word. |
| 1228 | - An unavailable app region gives C16, the allow-list C17, a store error 502, |
| 1229 | and a defaulted home inside its `replicas` C26. |
| 1230 | - **Dormancy versus app-derived defaults (codex R2-2):** |
| 1231 | - With no placement published, an explicit `uk` is accepted, as today. |
| 1232 | - An app-derived `uk` is refused with C16 `absent`. |
| 1233 | - The last UK box's file removed: an app-derived `uk` is refused. |
| 1234 | - The edges listing fails: an app-derived create gets 503 C28, and an |
| 1235 | explicit create is unchanged. |
| 1236 | - The switch off with an existing `uk` row: the fill still applies. |
| 1237 | |
| 1238 | **No stamp** (§13.1). A non-home instance created by this spec carries no |
| 1239 | `Steering` field. The record is byte-identical to one created by |
| 1240 | `instances create --region` before this spec. |
| 1241 | |
| 1242 | **The guess invariant** (codex R3-4). A property test over generated |
| 1243 | policies, candidate sets, missing coordinates, unmappable hints and |
| 1244 | edges-read failures asserts two things: |
| 1245 | |
| 1246 | - every non-null guess passes `instanceRegionAllowed(m.Regions, |
| 1247 | homeRegionIn(code))`; |
| 1248 | - a non-null guess exists whenever some filtered candidate is reachable by |
| 1249 | the order in §6.4. |
| 1250 | |
| 1251 | Named cases: |
| 1252 | |
| 1253 | - an allow-list of only `uk`, with the home's coordinates missing; |
| 1254 | - an unmappable hint against the same list; |
| 1255 | - an edges-read failure against the same list, which gives `null`. |
| 1256 | |
| 1257 | **Pools, end to end.** A pooled manifest gives `guess: null` and |
| 1258 | `app_default.reason: pooled`. |
| 1259 | |
| 1260 | **Fleets, end to end** (codex R4-1). For an app placed on `fleet/acme` with |
| 1261 | an unrestricted shared-pool manifest: |
| 1262 | |
| 1263 | - `/regions/guess` returns `null`; |
| 1264 | - a create that submits `uk` anyway produces a record whose region is |
| 1265 | `fleet/acme`, which pins why the guess is withheld; |
| 1266 | - a failed placement read gives 502 C27. |
| 1267 | |
| 1268 | **Service through the real path** (codex R4-3), as an integration test with |
| 1269 | two edges and a backhaul between them: |
| 1270 | |
| 1271 | - a non-home instance answers a request that enters through the other edge; |
| 1272 | - with no mesh key mirrored, the same request gets the unknown-host answer |
| 1273 | (`internal/edge/edge.go:2699`, `:1428`), and the Phase P gate (§17) treats |
| 1274 | that as a failure, not a pass. |
| 1275 | |
| 1276 | **App create.** |
| 1277 | |
| 1278 | - An old body from a person resolves by IP, then home. This needs `ip` passed |
| 1279 | through `regionReq` (codex R2-5). |
| 1280 | - A `timezone` body. |
| 1281 | - Explicit regions: valid, invalid (C18) and unarmed (C19). |
| 1282 | - `place` plus `region`. |
| 1283 | - The switch off, or on with an unnamed home. |
| 1284 | - A service principal gets no IP step. |
| 1285 | - Adoption: |
| 1286 | - with `@auto`, or a `timezone` but no `region`, it stores `""`; |
| 1287 | - with an explicit code, it stores the code; |
| 1288 | - **a region named `auto`, and a home named `auto`, both work and are |
| 1289 | distinguishable from `@auto`** (codex R2-3). |
| 1290 | |
| 1291 | **Browser.** |
| 1292 | |
| 1293 | - One region, or the switch off: the form is byte for byte today's. |
| 1294 | - `@auto` is labelled with the IP guess. |
| 1295 | - §9.5 needs a session and CSRF. |
| 1296 | - An explicit code records `chosen`. |
| 1297 | |
| 1298 | **Endpoints.** |
| 1299 | |
| 1300 | - `/regions` for an existing app matches today's golden in `regions` and |
| 1301 | `default`. |
| 1302 | - `app_default` covers each `reason`. |
| 1303 | - For an app storing `uk` and placed on `fleet/acme`: `region` is `uk`, |
| 1304 | `effective` is `fleet/acme`, and `deploy --region uk` passes while |
| 1305 | `deploy --region ie` is refused (codex R2-4). |
| 1306 | |
| 1307 | **Rollback list** (codex R2-6). A non-home app is not listed for any of: |
| 1308 | |
| 1309 | - its static apex; |
| 1310 | - an addr-backed record; |
| 1311 | - a replica; |
| 1312 | - a pooled record. |
| 1313 | |
| 1314 | A no-region instance record created after the app is listed as a possible |
| 1315 | discrepancy, worded as such. |
| 1316 | |
| 1317 | **Privacy.** With a canonical-host redirect configured, a guess POST's `tz` |
| 1318 | and `ip` appear in none of: |
| 1319 | |
| 1320 | - the `Location` header; |
| 1321 | - the redirect body; |
| 1322 | - any error body; |
| 1323 | - the captured logs; |
| 1324 | - any audit row. |
| 1325 | |
| 1326 | **CLI.** |
| 1327 | |
| 1328 | - Timezone detection with injected functions. |
| 1329 | - One output line per source. |
| 1330 | - C13a and C13b. |
| 1331 | - `deploy --region`: different is refused before anything ships, and equal |
| 1332 | is silent. |
| 1333 | - `canary --region` against an existing canary in another region is refused |
| 1334 | before any write or upload. |
| 1335 | - `canary --region uk` on a fleet-placed app, run twice: both runs refuse |
| 1336 | with C30 before any create, rechannel or upload. The first run creates |
| 1337 | nothing, so the second sees the same state (codex R4-2). |
| 1338 | - Tests use `carlosTestEnv` with a scratch `cmd.Dir`. |
| 1339 | - `20-cross` covers the CLI changes. |
| 1340 | |
| 1341 | ## 17. Rollout |
| 1342 | |
| 1343 | **Phase P: placement.** |
| 1344 | |
| 1345 | 1. **Before any roll.** |
| 1346 | - Audit production manifests for `regions` (§12). |
| 1347 | - Confirm each box publishes its AWS region. In particular, confirm the |
| 1348 | flagship's Ireland box publishes `region: eu-west-1` and no `placement`. |
| 1349 | Use `CARLOS_AWS_REGION` where it does not. |
| 1350 | - Set `CONSOLE_HOME_REGION` on every console that will turn the switch on. |
| 1351 | 2. **Stage the secrets key (R9).** Every box in a region that should take |
| 1352 | placed instances gets the deployment's `CARLOS_SECRETS_KEY` in |
| 1353 | `/etc/carlos/host.env`, from the deployment's secret store. On the flagship |
| 1354 | today that means the UK, US and AU boxes. This is infrastructure work in |
| 1355 | carloku-infrastructure, under the scoped-approval rule (AGENTS.md). Set |
| 1356 | `CARLOS_SECRETS_PUBKEY` on the console if it is not already set. |
| 1357 | 3. **Box release.** It carries the edges-file eligibility fields and |
| 1358 | `secrets_key_fp`. Roll it to every box. Then check that every region's |
| 1359 | files carry all of them, and that each `secrets_key_fp` equals the |
| 1360 | console's. Until then, no coded region is eligible (§6.1). |
| 1361 | 4. **Console release, switch off.** It carries the migration, `internal/geo`, |
| 1362 | the fill, the endpoints and the UI. Behaviour is unchanged, and it can be |
| 1363 | rolled back freely. |
| 1364 | 5. **Verify on dev.** Turn the switch on with two eligible regions. Then: |
| 1365 | - create an app in the browser, and read its row; |
| 1366 | - create an instance with no region, and read its record; |
| 1367 | - diff `/regions` for an old app against the pre-roll answer; |
| 1368 | - run the service gate below. |
| 1369 | 6. **Service gate on the flagship, with the switch still off** (codex R4-3). |
| 1370 | |
| 1371 | Eligibility proves configuration, not a working request path. |
| 1372 | Reachability is only a non-empty EIP (`instancesplacement.go:185`), and |
| 1373 | backhaul refuses to dial without a mirrored mesh key (`edge.go:2699`). |
| 1374 | So before the switch goes on, **every eligible region must serve a fresh |
| 1375 | instance through the real public path.** |
| 1376 | |
| 1377 | For each eligible region R, create one probe instance of a test app with |
| 1378 | an explicit `--region R`. That is ordinary API use and needs the switch |
| 1379 | off. Deploy a known build to it, then check: |
| 1380 | |
| 1381 | - **From outside, through the platform host:** `curl -s -D - -o /dev/null |
| 1382 | https://<probe host>/` from a machine outside the deployment, using |
| 1383 | public DNS as it stands. On the flagship that enters through the Ireland |
| 1384 | edge, so every non-home region is exercised over backhaul. Expect 2xx, |
| 1385 | and an `X-Carlos-Version` equal to the deployed build. |
| 1386 | - **Through an edge that is not the instance's box:** for every armed |
| 1387 | edge E other than R's box, `curl -s -D - -o /dev/null --resolve <probe |
| 1388 | host>:443:<E's EIP> https://<probe host>/`, with the same expectations. |
| 1389 | This covers the home region too, entered from a far edge, and every |
| 1390 | edge-to-box pair the static DNS could someday route. |
| 1391 | |
| 1392 | A 404 with the unknown-host answer, a TLS failure or a version mismatch |
| 1393 | fails the gate. **The switch is turned on only when every eligible region |
| 1394 | passes.** A region that fails is fixed first (mesh key, certificate, |
| 1395 | activator), or its boxes are made ineligible, before the switch goes on. |
| 1396 | The probe instances are deleted afterwards. The same gate is re-run before |
| 1397 | the switch is turned on again after any period off, and whenever a new |
| 1398 | region becomes eligible. |
| 1399 | 7. **Flagship.** Roll the console, then turn the switch on. **From here the |
| 1400 | binary rollback floor applies (R7).** That release's notes and the roll |
| 1401 | recipe in carloku-infrastructure name the floor version. Optionally, set |
| 1402 | `CONSOLE_GEOIP_DB` to a DB-IP Lite file. |
| 1403 | 8. **CLI release.** It is safe in either order (§9.6). |
| 1404 | 9. **titogo**, in its own repo, adopts `/regions/guess` and the place names. |
| 1405 | |
| 1406 | No step of Phase P depends on Phase L. Phase P writes nothing to DNS, stamps |
| 1407 | no `Steering` field, and needs neither `CARLOS_STEERING_ZONE_ID` nor any |
| 1408 | change to the zone. |
| 1409 | |
| 1410 | **Phase L: latency (R6, R8)** is the follow-on spec's rollout, not this one's |
| 1411 | (§13.3). Its gate is that spec landing with its simulation passing, plus |
| 1412 | scoped approval for the infrastructure. Until then, **no deployment arms |
| 1413 | `CARLOS_STEERING_ZONE_ID`**, including for the static-site steering that |
| 1414 | already exists. The latency baseline is taken once Phase P step 7 has run. |
| 1415 | That makes Phase L depend on Phase P, never the other way round. |
| 1416 | |
| 1417 | Copy review (§19) happens before Phase P step 4 ships any string. |
| 1418 | |
| 1419 | ## 18. Questions for Paul |
| 1420 | |
| 1421 | None. R8 settled rev 4's sequencing question. |
| 1422 | |
| 1423 | ## 19. Copy (pending copy review) |
| 1424 | |
| 1425 | These are drafts, and none ships until Paul has reviewed it through copy |
| 1426 | review. There are no em dashes. `{…}` is filled at runtime. |
| 1427 | |
| 1428 | | id | where | draft | |
| 1429 | | --- | --- | --- | |
| 1430 | | C1 | new-app page, picker label | Region | |
| 1431 | | C2 | new-app page, help | Where this app's instances run. We picked the region nearest to you. Choose another if most of your users are somewhere else. | |
| 1432 | | C3 | new-app page, `@auto` option | {place} (nearest to you) | |
| 1433 | | C5 | new-app page, flash | That region can't take new apps right now. Choose another. | |
| 1434 | | C7 | Instances tab | New instances start in {place}, this app's region. | |
| 1435 | | C8 | Instances tab, default unavailable | New instances can't start in {place}, this app's region, right now. Pass a region to place one elsewhere. | |
| 1436 | | C9 | CLI, from timezone | Region: {place} (from your timezone). Use --region to choose another. | |
| 1437 | | C10 | CLI, from IP | Region: {place} (from your network location). Use --region to choose another. | |
| 1438 | | C11 | CLI, fallback | Region: {place} (the platform default). Use --region to choose another. | |
| 1439 | | C12 | CLI, chosen | Region: {place}. | |
| 1440 | | C13a | CLI, old console ignored --region | This console does not record an app region yet, so --region was not applied. | |
| 1441 | | C13b | CLI, console did not apply --region | This console is not assigning app regions yet, so --region was not applied. This app's instances start in {place}. | |
| 1442 | | C14 | CLI, instances create and canary | created instance {host} (channel {channel}, {place}) | |
| 1443 | | C15 | CLI, deploy --region on an existing app | {app} already runs in {place}. --region only applies when deploy creates an app. To place one instance elsewhere, use carlos instances create --region. | |
| 1444 | | C16 | API, defaulted region unavailable | this app's region {code} can't take new instances right now ({reason}). Pass a region to place this instance elsewhere. Regions that can: {list}. | |
| 1445 | | C17 | API, defaulted region not allowed | this app's region {code} is not one its instances may use. Allowed: {list}. Pass a region to choose one. | |
| 1446 | | C18 | API, app create, unknown region | region {code} is not one this platform runs in. Choose from: {list}. | |
| 1447 | | C19 | API, app create, region not armed | region {code} can't take new apps right now. Choose from: {list}. | |
| 1448 | | C20 | API, app region unreadable | could not read this app's region. Try again. | |
| 1449 | | C21 | activity feed | created app {name} in {place} | |
| 1450 | | C22 | flag help | apps create: "region this app's instances start in (default: the region nearest you)". deploy: "region for the app this deploy creates (default: the region nearest you)". canary: "region for the canary instance (default: the app's region)". instances create: "region for this instance (default: the app's region, subject to the manifest's allow-list)". | |
| 1451 | | C24 | new-app page, IP step configured | IP location by DB-IP. (Linked to db-ip.com; DB-IP's required wording to be confirmed against its licence page.) | |
| 1452 | | C30 | CLI, canary --region on a fleet-placed app | {app} runs on your fleet {fleet}, which decides where every instance runs, so --region does not apply. Run the command without --region. | |
| 1453 | | C31 | API, explicit create into a region without the secrets key (R9) | region {code} can't run instances right now: its boxes don't hold this deployment's secrets key, so an instance there would start without its config or secrets. Choose another region. | |
| 1454 | | C25 | CLI, canary --region on an existing canary | canary {host} already runs in {place}. --region applies only when canary creates the host. Run carlos canary remove first to start it somewhere else. | |
| 1455 | | C26 | API, defaulted home inside its replica set | this app's region {code} is also one of this instance's replica placements. Pass a region for the instance, or drop {code} from replicas. | |
| 1456 | | C27 | API, guess with unreadable policy | could not read this app's instance policy, so there is no region suggestion. Try again. | |
| 1457 | | C28 | API, app region unconfirmed | could not confirm that this app's region {code} can take new instances. Try again, or pass a region. | |
| 1458 | | C29 | roll-forward log, read by operators | possible placement discrepancy: {host} of {app} has no region, but {app}'s region is {code}. It may be a deliberate home override; review it. | |
| 1459 | |
| 1460 | The `{reason}` words in C16 are `absent`, `not-hosting`, `no-secrets-key` |
| 1461 | (R9) and `not-eligible`. Shown to a person, they read: "no box runs there |
| 1462 | now", "boxes there can't start instances", "boxes there can't open this |
| 1463 | deployment's secrets" (R9), and "boxes there don't take ordinary instances". |
| 1464 | Those are copy too. |
| 1465 | |
| 1466 | C23, place names: |
| 1467 | |
| 1468 | | AWS region | place | AWS region | place | |
| 1469 | | --- | --- | --- | --- | |
| 1470 | | us-east-1 | US East · N. Virginia | eu-north-1 | Sweden · Stockholm | |
| 1471 | | us-east-2 | US East · Ohio | il-central-1 | Israel · Tel Aviv | |
| 1472 | | us-west-1 | US West · N. California | me-south-1 | Bahrain | |
| 1473 | | us-west-2 | US West · Oregon | me-central-1 | UAE | |
| 1474 | | ca-central-1 | Canada · Montréal | af-south-1 | South Africa · Cape Town | |
| 1475 | | ca-west-1 | Canada · Calgary | ap-south-1 | India · Mumbai | |
| 1476 | | mx-central-1 | Mexico · Querétaro | ap-south-2 | India · Hyderabad | |
| 1477 | | sa-east-1 | Brazil · São Paulo | ap-east-1 | Hong Kong | |
| 1478 | | eu-west-1 | Ireland · Dublin | ap-east-2 | Taiwan · Taipei | |
| 1479 | | eu-west-2 | UK · London | ap-southeast-1 | Singapore | |
| 1480 | | eu-west-3 | France · Paris | ap-southeast-2 | Australia · Sydney | |
| 1481 | | eu-central-1 | Germany · Frankfurt | ap-southeast-3 | Indonesia · Jakarta | |
| 1482 | | eu-central-2 | Switzerland · Zurich | ap-southeast-4 | Australia · Melbourne | |
| 1483 | | eu-south-1 | Italy · Milan | ap-southeast-5 | Malaysia | |
| 1484 | | eu-south-2 | Spain · Aragón | ap-southeast-6 | New Zealand · Auckland | |
| 1485 | | ap-northeast-1 | Japan · Tokyo | ap-southeast-7 | Thailand · Bangkok | |
| 1486 | | ap-northeast-2 | South Korea · Seoul | ap-northeast-3 | Japan · Osaka | |
| 1487 | |
| 1488 | The `docs/site/cli.md`, README and release-note additions (§5.7, §11, §12, |
| 1489 | §14) are prose, and go through the same review. |
| 1490 | |
| 1491 | ## 20. Follow-ups (not in this spec) |
| 1492 | |
| 1493 | - **Changing an existing app's region** (R5). An owner-only verb that affects |
| 1494 | future creates only and leaves existing instances alone. |
| 1495 | - **Phase L: latency routing (R6)**, on its own spec, meeting §13.2's |
| 1496 | invariants with a simulation over generated fleets. It derives "steer this |
| 1497 | host" from the record's region, which also covers moved instances. It |
| 1498 | carries #169's infrastructure plan, the delegation check for the flagship's |
| 1499 | `oncarlos.com` zone, and the latency measurement. |
| 1500 | - **Pool-aware candidates.** Pool eligibility proven by the target box itself. |
| 1501 | - **Sidecars following the app region**, with an armed gate on their door. |
| 1502 | - **A browser instance-create form.** |
| 1503 | - **Tightening #162** for explicit home-in-own-replica-set creates. |
| 1504 | |
| 1505 | ## 21. Review dispositions |
| 1506 | |
| 1507 | ### Round 1 (rev 1, 7fbc9479) |
| 1508 | |
| 1509 | | # | sev | finding | disposition | |
| 1510 | | --- | --- | --- | --- | |
| 1511 | | 1 | P1 | A picker submitting the guess bypasses the pool exclusion | **accepted.** No guess for a pooled policy (§6.1). | |
| 1512 | | 2 | P1 | Browser adoption silently changes an existing app's default | **accepted.** `regionReq` with guess resolution after adoption (§5.5). | |
| 1513 | | 3 | P1 | An unnamed home makes override and recovery impossible | **accepted.** Non-home writes need a named home (§5.6). | |
| 1514 | | 4 | P1 | No mixed-version or rollback rule | **accepted.** Read side first, the switch as lever, the binary floor (§5.7). Round 2 marked it "partly" pending acceptance of the floor; R7 settles that. | |
| 1515 | | 5 | P2 | `canary --region` undefined for an existing canary | **accepted** (§11). | |
| 1516 | | 6 | P2 | Manifest absence and read failure unspecified | **accepted.** Three policy states (§6.1, §9.3). | |
| 1517 | | 7 | P2 | Unavailable-default explanation disappears | **accepted.** `app_default` (§9.3), C8 (§10). | |
| 1518 | | 8 | P2 | Query hints reflected by redirects | **accepted.** POST bodies only (§6.5, §9.4, §9.5). | |
| 1519 | |
| 1520 | ### Round 2 (rev 2, bd4ed0c4) |
| 1521 | |
| 1522 | | # | sev | finding | disposition | |
| 1523 | | --- | --- | --- | --- | |
| 1524 | | R2-1 | P1 | Automatic candidates can include pool-restricted or no-inbound boxes that never mint shared instances | **accepted.** Two additive edges-file fields, positive eligibility for automatic choices, and a box release before the switch (§6.1, §17). I chose publication over a console-side operator declaration because the box is the only party that knows its own pools and inbound setting, and the `hibernation_configured` precedent is already there. | |
| 1525 | | R2-2 | P1 | Losing the last placement publication disables the unavailable-default refusal | **accepted.** App-derived defaults need positive eligibility and never ride dormancy. A read failure is distinct (503 C28). The contradicting test is replaced (§7, §16). | |
| 1526 | | R2-3 | P2 | `auto` collides with a valid placement code | **accepted.** The sentinel is `@auto`, outside the grammar, with tests for a region and a home named `auto` (§5.5, §10). | |
| 1527 | | R2-4 | P2 | Fleet `app_default.code` misleads the deploy preflight | **accepted.** `app_default.region` (stored) is separate from `effective` (the fleet itself), and deploy compares against `region` (§9.3, §11). | |
| 1528 | | R2-5 | P2 | `regionReq` cannot carry the IP `claimApp` needs | **accepted.** A request-local `ip` from `s.clientIP` in `regionReq`, resolved after adoption, never logged or stored (§5.5). | |
| 1529 | | R2-6 | P2 | The rollback detector flags correct deployments | **accepted.** Non-inheriting shapes are excluded, matches are worded as possible discrepancies for review, and the runbook records the interval (§5.7, C29). | |
| 1530 | |
| 1531 | ### Astra's recommendations on rev 1's open questions |
| 1532 | |
| 1533 | | rev 1 Q | disposition | |
| 1534 | | --- | --- | |
| 1535 | | Q1 titogo adoption in "done" | superseded by R5: done is new apps plus moves | |
| 1536 | | Q2 placement-only, measure first | superseded by R6: latency is in scope. Measurement moves to the follow-on spec (§13.3). | |
| 1537 | | Q3 refuse an unavailable default | adopted (§7) | |
| 1538 | | Q4 creation-only region | adopted; the change verb is a follow-up (R5, §20) | |
| 1539 | | Q5 defer the browser instance form | adopted (§20) | |
| 1540 | | Q6 DB-IP Lite, corrected GeoLite2 note | adopted (§6.2, C24) | |
| 1541 | | Q7 pinned reader and fixtures | adopted (§6.2) | |
| 1542 | | Q8 picker style, geographic consistency | adopted (§14) | |
| 1543 | | Q9 verify home publication, `CARLOS_AWS_REGION` | adopted (§17) | |
| 1544 | | Q10 sidecars unchanged | adopted (§20) | |
| 1545 | | Q11 suppress pooled guesses | adopted (§6.1) | |
| 1546 | | Q12 home inside own replica set | adopted for defaulted regions (§7); explicit is a follow-up (§20) | |
| 1547 | | Q13 keep allow-list normalization, audit | adopted (§12, §17) | |
| 1548 | | Q14 unnamed home | adopted, as finding 3 | |
| 1549 | | Q15 Windows fallback | adopted (§11) | |
| 1550 | | Q16 AWS list | adopted (§14) | |
| 1551 | |
| 1552 | ### Astra's round-2 recommendations on rev 2's questions |
| 1553 | |
| 1554 | | rev 2 Q | disposition | |
| 1555 | | --- | --- | |
| 1556 | | Q1 (b): titogo in "done" | superseded by R5 | |
| 1557 | | Q2 (a): placement-only, measure first | superseded by R6. Measurement moves with Phase L to the follow-on spec (§13.3). | |
| 1558 | | Q3 (a): accept the floor | matches R7 | |
| 1559 | |
| 1560 | ### Round 3 (rev 3, 8e59f598) |
| 1561 | |
| 1562 | | # | sev | finding | disposition | |
| 1563 | | --- | --- | --- | --- | |
| 1564 | | R3-1 | P1 | Arming per-host steering breaks DNS for unsteered siblings, through empty non-terminals shadowing `*.<apps-domain>` | **accepted.** Verified against `steering.go:369`, `:386` and carloku's single wildcard (`dns53.tf`, `dns.tf`). It already affects static-site steering. Resolved by invariant I1, the gate on arming #169, and the move of Phase L to a follow-on spec (§13). | |
| 1565 | | R3-2 | P1 | One latency record per edge is invalid when two edges share an AWS region | **accepted.** Verified at `route53.go:597`. Resolved by invariant I2, which groups by AWS region, and a Route53 fake that enforces the rule (§13.2). This departs from the coordinator's example wording "keyed by edge, not region", because Route53's constraint is per region. | |
| 1566 | | R3-3 | P1 | The `CARLOS_POOLS=,tito` test would publish shared eligibility the reconciler denies | **accepted.** Verified at `agent.go:2033`. Publication is computed from the same `ReconcileConfig` and predicate, `,tito` publishes false, and a test pins the equality (§6.1, §16). | |
| 1567 | | R3-4 | P2 | Home fallback is not constrained by the allow-list | **accepted.** The guess invariant, with the home fallback only when the home passes the policy, and a property test (§6.4, §16). | |
| 1568 | | R3-5 | P2 | Large DNS withdrawals are refused indefinitely, not spread over passes | **accepted.** Verified at `route53.go:207` and `steering_loop.go:154`. The wrong claim is removed, and invariant I4 makes it the follow-on spec's requirement (§13). | |
| 1569 | | R3-6 | P3 | §4 reverses which requests need positive eligibility | **accepted.** Corrected to app-derived (§4). | |
| 1570 | |
| 1571 | ### Astra's round-3 recommendations |
| 1572 | |
| 1573 | | former question | disposition | |
| 1574 | | --- | --- | |
| 1575 | | Q1: apply R5, titogo separate | adopted (§2, §17 step 9) | |
| 1576 | | Q2: implement R6 with the DNS corrections, keep the separate approved Phase L rollout and measurements | adopted in substance. The corrections are larger than a patch, so they move to the follow-on spec with invariants (§13); the sequencing is ruled by R8. | |
| 1577 | | Q3: name the exact floor version before activation; keep region-aware reads when disabling | adopted (§5.6, §5.7, §17 step 7) | |
| 1578 | |
| 1579 | ### Round 4 (rev 4.1, 4ad64f7c) |
| 1580 | |
| 1581 | | # | sev | finding | disposition | |
| 1582 | | --- | --- | --- | --- | |
| 1583 | | R4-1 | P2 | Fleet-placed apps still get instance guesses the door will override | **accepted.** Verified at `fleets.go:479` and `apiinstances.go:798`. The guess endpoint resolves fleet placement first and returns `null`; app-creation inference is unchanged; end-to-end test (§6.1, §9.4, §12, §16). | |
| 1584 | | R4-2 | P2 | `canary --region` on a fleet behaves differently before and after creation | **accepted.** Verified at `canary.go:405` and `apiinstances.go:798`. Fleet precedence is resolved before either branch, and `--region` on a fleet-placed app refuses identically every time with C30, never suggesting a remove (§11, §12, §16). | |
| 1585 | | R4-3 | P2 | Phase P verification can pass while non-home instances are unreachable | **accepted.** Verified at `instancesplacement.go:185`, `edge.go:2699` and `:1428`. A service gate before the production switch: a fresh probe per eligible region, reached from outside through the platform host and through every other edge with `--resolve`, checking 2xx and `X-Carlos-Version`. All regions must pass. Plus an integration test for the missing-mesh-key case (§16, §17 step 6). | |
| 1586 | |
| 1587 | Astra raised no open questions in round 4. §18 stands at none. |
| 1588 | |
| 1589 | ### Amendment rev 6.2 (`instance_domain`, Paul 2026-10-03) |
| 1590 | |
| 1591 | Not a review finding of this spec. Astra's plan round 3 (finding 1) showed |
| 1592 | that `canary --region`, deciding from the capped instance list, could miss |
| 1593 | an existing canary and write. The plan answered with `instance_domain` and a |
| 1594 | single-record read. Astra's plan round 4 judged it sound, including the |
| 1595 | check of the PUT's answer against the preflight, and Paul approved it. §9.3 |
| 1596 | carries it, with the round-4 consequence of R9a for pooled and fleet apps. |
| 1597 | |
| 1598 | ### Amendment R9a (rev 6.1, Paul 2026-10-02) |
| 1599 | |
| 1600 | Not a review finding of this spec. Astra's review of the implementation |
| 1601 | plan (plan round 2, finding 3) showed that a 200 marked `policy-unreadable` |
| 1602 | let the CLI read an unread region as the home. Paul accepted the two flags |
| 1603 | that fix it. Astra's plan round 3 judged them sound. §3 (R9a) and §9.3 carry |
| 1604 | them, and the copy record gains C15c for deploy's refusal. |
| 1605 | |
| 1606 | ### Amendment R9 (rev 6, Paul 2026-10-02) |
| 1607 | |
| 1608 | Not a review finding: a production finding, ruled by Paul. Recorded here so |
| 1609 | the dispositions list every change since rev 5. The changes are §3 (R9), |
| 1610 | §6.1 (`secrets_key_fp`, its format, the console's expected fingerprint, the |
| 1611 | fourth eligibility fact), §7 and §8.1 (the explicit refusal, C31), §9.3 and |
| 1612 | C16 (the `no-secrets-key` reason), §15, §16, §17 (staging the key), and §19 |
| 1613 | (C31). |
| 1614 | |
amadan