carlos / platform Public
Nearest region by default: each app picks its region at creation
Date: 2026-10-01 Revision: 6.2.
- Rev 1 (7fbc9479) went to Astra (GPT-6, via codex), and the findings are in
CODEX-FINDINGS-NEAREST-REGION-R1.md. - Rev 2 (bd4ed0c4) went back, and the findings are in
CODEX-FINDINGS-NEAREST-REGION-R2.md. - Rev 3 (8e59f598) went back again, and the findings are in
CODEX-FINDINGS-NEAREST-REGION-R3.md. - Every finding of all three rounds was checked against the code and accepted. Changes are marked "(codex N)", "(codex R2-N)" and "(codex R3-N)", and §21 records each disposition.
- Rev 3 folded in Paul's answers to rev 2's questions as rulings R5 to R7.
- Rev 4 moves latency routing (R6) into a follow-on spec, because the existing steering converger is not safe to arm (§13). That spec gets invariants to meet, and placement ships first.
- Rev 4.1 records Paul's ruling on that split as R8 and removes the question.
- Rev 5 answers Astra's round 4 (
CODEX-FINDINGS-NEAREST-REGION-R4.md): three P2s, marked "(codex R4-N)". Astra's round 5 found nothing new; Paul approved rev 5 on 2026-10-02. - Rev 6 is an amendment Paul approved on 2026-10-02, ruling R9, after a production finding: only the Ireland box holds
CARLOS_SECRETS_KEY. On the UK, US and AU boxes the config loops log "config sync loop disabled: CARLOSSECRETSKEY not set" (internal/host/agent.go:881-886), and a wake treats a missing<host>.config.envas empty (internal/activator/backend_exec.go:1146-1149). An instance placed there runs with no config, no secrets and no store binding. That was part of the kpmg outage. Changes are marked "(R9)". - Rev 6.1 records ruling R9a (Paul, 2026-10-02):
app_defaultgainsregion_knownandplacement_known. They are additive, and absent knowledge fails closed. They came out of the implementation plan's review (Astra plan round 2, finding 3), and §9.3 states their semantics. - Rev 6.2 records
instance_domain(Paul approved it on 2026-10-03): an additive, read-only field ofapp_default, the authoritative hostname suffixcanary --regionreads its own record under before any write (§9.3). It came out of Astra's plan round 3 (finding 1), and her round 4 judged it sound. Rev 6.2 also writes down a consequence of R9a that round 4 found (finding 1): with the stored region unread,effectiveis empty even for a pooled or fleet-placed app, socanary --regionneeds both flags.
Status: Approved by Paul on 2026-10-02: rev 5, plus the R9 amendment in this revision. The rulings in §3 are Paul's and are not reopened here. Discussion: not yet filed on amadan. Latency routing depends on amadan #169. Builds on:
- #329, the armed-placement gate
- #346, the home region's name (
CONSOLE_HOME_REGION) - the customer-fleets placement column
- #162, replica placements
- #175/#331, instance moves
- the multi-region steering converger (
internal/steering)
1. The problem, as found
On 2026-10-01 the Ireland box carried 116 of the flagship's 116 non-static instances and reached 91% disk. Overnight, one customer's three instances crash-looped for 4h18m on a full disk. Nothing chose Ireland. Nothing chose anything.
Every create path ends at one door, and that door's default is the home box:
- The instance create door,
handleAPIInstanceCreate(internal/console/apiinstances.go:608), trimsreq.Regionand hands it tohomeRegionIn(apiinstances.go:772; the function is atinternal/console/apiinstancemove.go:136). The home's name normalizes to"", and""is the unconstrained record only the home box mints. - The new-app page (
internal/console/templates/app_new.html) asks for a name and nothing else. The Instances tab's Placement section (templates/app_instances.html:60-78) offers "Shared fleet" or one of the account's own fleets, which is a different axis. carlos deploycreates an app only throughofferCreateApp(deploy.go:513, call at:536), and sends no region. Its only instance write is the static platform host (mintPlatformHost,deploy.go:850, create at:858), which sends"".carlos canarycreates its host atcanary.go:407with region"".- Only
carlos instances create --region(instances_cli.go:293, sent at:322) chooses, and only when a person remembers to. External signup paths (titogo, Managed Jelly) hold instance-role tokens and send no region.
And nothing is steered. Latency routing exists (internal/steering), but it is armed nowhere, because no deployment sets CARLOS_STEERING_ZONE_ID (amadan #169). Every instance host therefore resolves through the deployment's static DNS. On the flagship that is one *.oncarlos.com record pointing at the Ireland EIP, served by DNSimple. The Route53 copy of the zone is staged and is not yet authoritative (carloku-infrastructure dns.tf and dns53.tf).
What exists to build on:
- Placement codes are bare (
ie,uk,au):CARLOS_REGIONverbatim. - Place names: the only one anywhere is
STATUS_BOX_REGION, operator free text for the public status page (internal/host/agent.go:88-92). - The edges file: every box publishes its AWS region and its placement code there (
internal/host/edgesfile.go:155-176). The region comes fromedgesFileRegion(:537), which honours aCARLOS_AWS_REGIONoverride. - What the console already reads: both of those, through
edgesFilePlacementView(internal/console/instancesplacement.go:35-55) andservedBy.Region(:25-28). - Capability:
placementsandarmedPlacements(instancesplacement.go:173,:207) say which codes can host. - The regions endpoint:
GET /api/cli/apps/{account}/{app}/regions(internal/console/apiregions.go:31-55) returns codes,armedand"default": HomeRegion, and reads no manifest.
There is no geolocation of any kind.
2. Goals and non-goals
Goals:
- A new app gets a region at creation, preselected from a guess of where its creator is. Its new instances start there unless told otherwise.
- A new instance placed outside the home is reached through the nearest edge, not through the home's (R6). The follow-on spec for Phase L delivers this goal, against the invariants in §13.
- The guess is generic: any deployment, any set of AWS regions.
- The guess can never name a region the create door would refuse, or one no box would mint (codex 1, codex R2-1).
- Existing apps, older CLIs and external callers that send nothing behave exactly as today.
Done has two stages:
- This spec is done when new apps get regions and new instances follow them.
- The work as a whole (R5, R6) is done when the follow-on spec has made latency routing safe to arm, and #169 is armed.
Existing apps keep their region. Relieving Ireland of the instances it already holds is the instance-move work (#175/#331), not this spec.
Non-goals, each with a follow-up in §20:
- Arming DNS steering and changing the converger. This is Phase L, the follow-on spec (§13).
- Moving existing instances.
- Changing an existing app's region (R5).
- A browser instance-create form.
- Sidecars following the app's region.
- Pool-aware candidates.
- Box sizing and disk alerts.
- An account-level default (R1).
3. Rulings (Paul, 2026-10-01)
| # | Ruling | Where this spec spends it | | --- | --- | --- | | R1 | Region is both latency and placement, chosen per app. Each app picks its region at creation, preselected from a location guess. No account-level default. | §5, §8.1, §13 | | R2 | The app's region is the default for new instances. It can be overridden per instance (picker or --region), subject to the manifest Regions allow-list and the armed/hosting check. | §7 | | R3 | Guess timezone first. Fall back to IP geolocation when the timezone is UTC, Etc/* or otherwise unmappable. | §6 | | R4 | The CLI guesses too. carlos deploy creating an app sends its timezone, the console resolves it, and the CLI prints one line ("Region: UK · London (from your timezone). Use --region to choose another."). No prompt. --region overrides everywhere. carlos instances create defaults to the app's region. | §11 | | R5 | Done = new apps only, plus moves. Existing apps keep their region. Relieving Ireland comes from instance moves. An owner-only verb to change an existing app's region is out of scope (a follow-up). | §2, §20 | | R6 | Latency routing is in scope for non-home instances, not placement alone. | §13: its invariants and gates, delivered by the follow-on spec | | R7 | The rollback floor is a runbook rule. It is documented for release notes and the roll recipe, with no outside guard. | §5.7, §17 | | R8 | Split (Paul, 2026-10-01). Placement ships on this spec. Latency routing (R6) is sequenced to its own follow-on spec, built to the §13 invariants. | §13, §17 | | R9 | A box must hold the deployment's secrets key to receive placed instances (Paul, 2026-10-02). | §6.1, §7, §8.1, §17 | | R9a | app_default gains region_known and placement_known (additive; absent knowledge fails closed) (Paul, 2026-10-02). | §9.3 | | D1 | The app region is stored console-side and set once at creation. The create door fills an empty req.Region from it before normalization, the allow-list and the armed checks. Existing apps are backfilled to the home, so behaviour does not change. | §5, §7 | | D2 | The guess works in coordinates: timezone to a point, IP to a point through an operator-supplied MaxMind-format .mmdb (CONSOLE_GEOIP_DB), then the great-circle nearest eligible region. Home when everything fails. | §6 | | D3 | Surfaces: a new-app picker; names and the guess for external pickers; the CLI as R4. | §9, §10, §11 | | D4 | Place names come from one platform-shipped AWS-region table, used everywhere and reconciled with STATUS_BOX_REGION. | §14 |
4. The shape, in one paragraph
The console's apps row gains two columns:
region, in the record's own vocabulary, where""is the home;region_source, one word saying how it was picked.
Both app-create doors resolve a region before the row is written. The order is an explicit choice, then the creator's timezone, then the request's IP (through the optional GeoIP database), then the home. Resolving means finding the great-circle nearest region whose boxes have positively said they can mint an ordinary instance.
From then on, an instance create that names no region gets the app's region, and that value meets every check an explicit one meets. An app-derived value additionally needs that positive eligibility, while an explicit one keeps today's gate (codex R3-6).
Latency routing for non-home instances (R6) is specified as invariants (§13) and delivered by a follow-on spec. Until that lands, this spec writes nothing to DNS and stamps nothing.
Existing rows read "", which is what they have always meant. Nothing writes a non-home app region until the operator turns the feature on, and the console will not turn it on without a named home.
5. Data model
5.1 Columns
(decision) Two columns, appended to the ALTER list in migrate (internal/console/store.go:91, list ending :386):
ALTER TABLE apps ADD COLUMN region TEXT NOT NULL DEFAULT '';
ALTER TABLE apps ADD COLUMN region_source TEXT NOT NULL DEFAULT '';regionis a placement code in the record vocabulary:""is the deployment's home, exactly as on aprovision.Record(#346). A choice of the home's name is normalized withhomeRegionInbefore storing, and displays usehomeRegionOut(apiinstancemove.go:143).region_sourceischosen,timezone,ip,default, or""(the row predates this feature). It is a label for people and logs, and nothing branches on it.
Nothing else is stored: not the timezone, not the IP, not a coordinate.
5.2 The rebuild trap
migrateAppsKey (store.go:472) rebuilds apps with a fixed column list (:487-505) and runs after the ALTER list. On a database not yet rebuilt, the new columns would be added and then dropped by the rebuild's INSERT … SELECT, and every later read of region would fail.
Both columns go into apps_new's DDL and the INSERT's column list in the same change. This is the rule the comment at :499-503 already records for placement, control_plane and build_probe.
5.3 Written with the row
(decision) The region rides the INSERT that claims the name.
- A new
CreateAppInRegion(accountID, app, createdBy, region, source, now)carries both columns. CreateAppIn(store.go:1349) becomes a call to it with("", ""), soCreateApp(:1342) is unchanged.claimApp(internal/console/server.go:3466) gains aregionReqparameter (§5.5).
This departs from the placement precedent on purpose. SetAppPlacement after CreateAppIn (server.go:3550-3567) is best-effort, because carlos apps place can repair it. An app region has no repair verb (R5), so a failed second write would leave the app at the home for good, silently.
Rename (store.go:1791) and transfer (:1709) update the same row, so the region survives both. The legacy owners backfill (:665) inserts defaults, which mean the home.
5.4 Backfill
(decision) The column default is the backfill. Every existing row reads ""/"". "" is the record vocabulary's home, so the door's fill (§7) turns "" into "" and nothing changes, byte for byte.
Writing the home's code instead would be worse on every count:
- It would gain nothing, because
homeRegionInnormalizes it back to"". - It would be impossible where
CONSOLE_HOME_REGIONis unset. - It would freeze the home as of migration day, where
""follows the home as records already do.
The backfill is idempotent by construction: the ALTERs swallow "duplicate column", and there is no one-shot PRAGMA user_version step.
5.5 Explicit choice versus guess, and adoption (codex 2, codex R2-3, codex R2-5)
claimApp receives a request-local struct, not a resolved region:
type regionReq struct {
explicit string // a person's choice: a code or the home's name; "" = none
tz string // a hint
ip net.IP // from s.clientIP, set only for a person's request
}Only explicit is a choice. The guess is resolved inside claimApp, after it knows whether this is an adoption, so no caller can pass a guess off as a choice.
ip comes from s.clientIP (internal/console/authcli.go:876). claimApp has no *http.Request, and its addr argument is the audit actor (apicli.go:349), not a network address (codex R2-5). Both callers fill ip before the call. It lives only for that call, and is never logged or stored.
(decision) Adoption never changes an existing app's region by guessing. An owner claiming a name whose objects already exist (adoptingExistingApp, server.go:3502, :3545) is restoring an app that probably has instances at the home. An adoption stores "" with source default unless explicit is set. A guess, however it was computed and whatever the form displayed, is discarded.
The browser form keeps the two apart without trusting any label. The preselected option's value is @auto, which is outside the placement-code grammar (^[a-z0-9-]{0,16}$, provision.go:688). A deployment with a region, or a home, named auto therefore still works (codex R2-3). Only a region code is explicit (§10). The CLI sends region only for --region, and sends timezone separately (§9.1).
5.6 The switch and the named home (codex 3, codex 4)
(decision) Non-home app regions are written only when both of these hold:
CONSOLE_APP_REGIONS=on(new; off by default);CONSOLE_HOME_REGIONis set.
If the switch is on and the home is unnamed, the console logs that once at startup and behaves as if the switch were off.
Without a named home, "" is the only way to say "home", and "" also means "no preference". No request could then choose the home over a non-home app default. A named home makes "region": "<home name>" an explicit choice at every door.
While the switch is effectively off:
- Every new app gets
""/default. An explicit non-homeregionon app create is not stored, and the response shows what was (§9.1). - The new-app picker is hidden.
- The guess endpoint (§9.4) still answers, because a per-instance guess for titogo writes nothing to the app row.
- The door's fill still runs. Turning the switch off stops new promises. It never breaks the ones already made.
5.7 Mixed versions and rollback (codex 4, R7)
The console owns its SQLite database, so one database is served by one console at a time. Mixed versions therefore means a rollback, or a roll that has not yet reached a deployment's console. A console older than this feature ignores the columns, so after a rollback it would land a no-region create for a uk app at the home.
(decision, under R7) The rule:
- Roll the read side first. The first release carries the migration, the fill, the endpoints and the UI, with the switch off. Every app row is then
"", so the release can be rolled back freely. - The switch is the feature's rollback lever. It is turned on only after that release has run cleanly. Turning it off stops new non-home values and keeps the fill.
- Once any app holds a non-home region, the binary rollback floor is the first region-aware release. This is a runbook rule (R7), with no outside guard. It is written in three places: - the README's "Releasing the platform" section (
README.md:227); - that release's notes; - the console roll recipe incarlosframework/carloku-infrastructure.
Rolling below the floor is out of policy. If it happens anyway, no-region creates land at the home, as they do today. Nothing is lost.
- On roll-forward, the console logs possible placement discrepancies (codex R2-6). For each app with a non-home region, it lists records that carry an empty region and were created after the app.
Records that never inherit the app default are excluded: static, addr-backed, replica, pooled and fleet-placed. Those are the shapes §7 leaves unfilled. A deliberate home override also produces an empty region, and the record keeps no provenance, so the list is a set of candidates for a person to review, not a detection of rollback damage. The runbook records the rollback interval, so the review can be narrowed to records created inside it.
CLIs mix freely in both directions (§9.6).
6. Resolving a region
6.1 Candidates and eligibility (codex R2-1)
(decision) One function, regionCandidates(ctx, policy), in instancesplacement.go, built from the same edges-file read placements uses. Each candidate is {code, awsRegion, eligible}.
Eligibility is positive and per box. Hosting checks reachability and the activator, but not whether a box will mint an ordinary instance:
- A box with
CARLOS_POOLSset serves the shared pool only if the list names it (poolServed,internal/provision/reconcile.go:1339). - A box with
CARLOS_INBOUND=nonerefuses every routed instance (reconcile.go:925).
Neither fact is in the edges file today. (decision) Each box publishes two additive fields in edgesFile (internal/host/edgesfile.go:155-176):
serves_shared_pool:poolServed(cfg.Pools, "");accepts_inbound:cfg.Inbound != InboundNone.
Both are computed from the very provision.ReconcileConfig the reconcile pass uses, which provisionTickConfig builds (internal/host/agent.go:2030). They are never re-parsed from the environment (codex R3-3). That parser drops empty elements, so CARLOS_POOLS=,tito is ["tito"] and does not serve the shared pool. The box publishes false for it. A box that should serve both the shared pool and a named one needs a parser change with its own tenancy tests, which is not part of this spec.
They follow hibernation_configured's precedent: not omitempty, so absent means "unknown" and false means "no". This is the one box-side change in this spec. It is purely additive, and old readers ignore it.
A code is eligible when at least one reachable box publishing it has hibernation_configured, serves_shared_pool and accepts_inbound all positively true. Unknown is not yes here. That is the reverse of placements' generosity, and deliberately so: that generosity is right for an explicit request, which a person made and can see refused. It is wrong for an automatic one.
Until boxes roll with the fields, no coded region is eligible. The guess then picks only the home, so the feature cannot outrun the box roll.
The secrets key (R9). An exec instance's config, secrets and store binding reach it only through <host>.config.env, which the agent renders only when the box holds CARLOS_SECRETS_KEY (agent.go:881-886, :1812). A wake treats a missing file as empty (backend_exec.go:1146-1149), so a box without the key runs a placed instance with nothing, silently. Eligibility therefore gains a fourth fact.
- What a box publishes. One more edges-file field,
secrets_key_fp: the fingerprint of the public half of the box'sCARLOS_SECRETS_KEY, never the key. The name, its place besideplacementanddata_bucket, and its shape (additive,omitempty, absent when the box holds no key or a key that does not parse) are the move-any-app spec's (2026-10-02-move-any-app-design.md§6, ondesign-moves-any-regionat e41c4fef). Both features read one field. - The format, which that spec names but does not fix, is fixed here for both:
"sha256:"followed by the lowercase hex SHA-256 of the public key's uncompressed SEC1 bytes ((*ecdh.PublicKey).Bytes(), 65 bytes for P-256). Those are exactly the bytesCARLOS_SECRETS_PUBKEYbase64-decodes to, so a box (from its private key) and the console (from the public key) compute the same string. One helper,appconfig.KeyFingerprint, is the only code that renders it. The move-any-app spec should cite it rather than define a second one. - What the console expects. The console learns the deployment's fingerprint from its own
CARLOS_SECRETS_PUBKEY(Config.SecretsPub,cmd/console/main.go:467-490). That is the key the CLI seals config to, so it is by definition the one a box must be able to open. The move-any-app spec's console makes the same choice. The console never learns the fingerprint from a box: a "majority of boxes" or "the home box" rule would certify whatever key happened to be widespread, including a wrong one. WithCARLOS_SECRETS_PUBKEYunset, the console cannot judge. It logs that once at startup, and no coded region is eligible for automatic placement. - The rule. A code is eligible only if at least one box that meets the other three facts also publishes a
secrets_key_fpequal to the console's. The home stays exempt, as it is from every eligibility fact: it is where every legacy create already lands, and on the flagship it is the one box that holds the key. - The reason word. A code that fails only this fact reports
no-secrets-key, betweennot-hostingandnot-eligiblein the order of checks (§9.3, C16).
Candidate rules:
- Coded regions: boxes whose
Placementpassesprovision.ValidRegion. Fleet codes (fleet/<name>) fail on the/. - The home: boxes that publish no placement, or the home's name, and have a dial address. The home is exempt from the positive check, because it is where every legacy create already lands.
handleAPIRegionsalready treats it as always armed (apiregions.go:42-47). - AWS region: the
Regionof the eligible boxes behind the code. If they disagree, the most common wins, ties go to the lexically smallest, and the disagreement is logged once. - Policy, read with
provision.GetManifestdirectly (codex 6). This is neverinstanceManifest, which answers 409 or 502 for the caller (apiinstances.go:117). - absent: no allow-list and no pool. - known: candidates pass throughinstanceRegionAllowed(apiinstances.go:1797) on the normalized code. A pooled manifest yields no candidates and no guess (codex 1). - unreadable: no guess, and an explicit error at the guess endpoint. - Fleet placement (codex R4-1). The instance guess endpoint (§9.4) resolves
appPlacement(internal/console/fleets.go:479) first. A fleet-placed app gets"guess": null, because the door replaces any submitted region with the fleet (apiinstances.go:798), so a guess could only mislead a picker. A failed placement read is an explicit error there (C27), never an unrestricted guess. App-creation inference (§5.5) is unaffected: it chooses the stored app region, which stays dormant while the app is placed. - Read failure (codex R2-2): the listing failing is distinct from "no file publishes a placement".
readEdgesFiles(instancesplacement.go:83) swallows the first today. A newreadEdgesFilesStrictreturns the error. Candidates built after a failed read are the home alone, if the policy admits it (§6.4), and the door treats the failure as unconfirmed (§7).
6.2 Points
A point is a latitude and longitude held in memory for one request.
Timezone. (decision) A new package, internal/geo, embeds three files from one IANA tzdata release:
zone.tab, one row per country and zone, with ISO 6709 coordinates;- a link table generated from tzdata's
Linklines; - a
VERSIONstamp.
zone.tab is used rather than zone1970.tab because browsers report names that zone1970.tab has merged away. For example, Europe/Amsterdam has its own row in zone.tab and none in zone1970.tab. The links matter because Chrome reports Asia/Calcutta where Firefox reports Asia/Kolkata. Go's time/tzdata carries no coordinates, so it cannot be the source.
The files are kept current by scripts/update-tzgeo.sh <version>, which fetches the release from data.iana.org, checks its signature, and rewrites the three files. Running it is a release-checklist step. A stale table only misses zone names newer than it, and those fall through to IP.
Lookup:
- A shape check: at most 64 bytes, one to three
/-separated segments of[A-Za-z0-9_+-]. - Unmappable by rule:
"",Local,UTC,UCT,GMT,Universal,Zulu,Factoryand all ofEtc/. - Links, at most 4 hops.
zone.tab, case-insensitive.
A miss at any step makes the timezone unmappable.
IP. (decision) CONSOLE_GEOIP_DB names an operator-supplied .mmdb. It is read in cmd/console/main.go beside CONSOLE_HOME_REGION (main.go:278) into a new Config.GeoIP.
- Recommended source: DB-IP Lite (CC BY 4.0). Its attribution appears on the page that uses it (C24) and in the README. MaxMind GeoLite2 also works, under its EULA's conditions, which include attribution and keeping the data updated. The operator owns that compliance.
- Unset: the IP fallback is off, which is the default.
- Unopenable: logged once, the fallback is off, and the console still starts.
- Lifecycle: the file is opened once and memory-mapped. Replacing it is an atomic rename plus a console restart.
- Lookup: the
locationfield, elsecountry.iso_code, elseregistered_country.iso_code, through country points. A lookup error is a miss, logged at most once a minute. - Never looked up: loopback, private, link-local, CGNAT and unspecified addresses.
The reader is github.com/oschwald/maxminddb-golang (ISC), pinned to a reviewed version. The tests use pinned fixtures from MaxMind's MaxMind-DB test data (MIT or Apache-2.0).
Country points. (decision) These come from the same zone.tab, not a second dataset.
- A one-row country uses its row.
- A multi-row country uses a shipped override naming its principal zone:
US→America/New_York,AU→Australia/Sydney,CA→America/Toronto,RU→Europe/Moscow,BR→America/Sao_Paulo,CN→Asia/Shanghai, and so on. - A test fails when a tzdata refresh gives a country a second row with no override.
Whose IP. The client IP from s.clientIP reaches the guess only for a person's request (§5.5). A service or app principal is a server, so it gets the IP step only by sending an address in the guess body (§9.4).
6.3 Nearest
Each eligible candidate's AWS region maps to a point through the region table (§14). Distance is haversine (mean Earth radius 6371.0088 km), rounded to whole kilometres.
- A candidate with no point is left out and logged once per code.
- If the home has no point, the guess is
default. - Ties go to the home, then to the lexically smallest code.
- An ineligible candidate is never picked.
6.4 Order
explicit region (validated as in §8.1) → "chosen"
else timezone gives a point → nearest → "timezone"
else IP gives a point → nearest → "ip"
else home → "default"The guess invariant (codex R3-4): every non-null guess, including every fallback to the home, passes the same normalized policy predicate the door applies (instanceRegionAllowed on the homeRegionIn form). The steps run in this order:
- Filter candidates by policy (§6.1).
- Pick the nearest among the filtered candidates that have a point.
- If the home has no point, or nothing maps, the fallback is the home, but only if the home is itself a filtered candidate.
- Otherwise the answer is
"guess": null.
So a manifest that allows only uk, with the home's coordinates missing, gives uk when the hint maps and null when it does not, never the home. There is also no guess when the policy is pooled or unreadable.
6.5 Logged, and not logged (codex 8)
- One line per app creation:
console: app region <sqid>/<app>: <code> (<source>). - Once per process: an unusable
CONSOLE_GEOIP_DB; a code with no point; boxes disagreeing on a code's AWS region.
The timezone, the IP and any coordinate never reach a URL, a log line, an audit row, a response body or the database. They travel only in POST bodies. The canonical-host redirect copies the whole query string into its Location header and its HTML (server.go:1306, :1326). A 308 on a POST keeps the body out of both.
7. The instance create door
(decision) The fill, replacing apiinstances.go:772:
raw := strings.TrimSpace(req.Region)
regionFrom := ""
if raw != "" {
regionFrom = "request"
}
if raw == "" && placed == "" && m.Pool == "" &&
strings.TrimSpace(req.Role) == "" &&
(provision.Record{Kind: strings.TrimSpace(req.Kind), Addrs: req.Addrs}).DerivesPaths() {
appRegion, _, err := s.store.AppRegion(acct.ID, app)
if err != nil {
// log; 502 C20
}
if appRegion != "" {
raw, regionFrom = appRegion, "app"
}
}
region := s.homeRegionIn(raw)Where it sits. After appPlacement (apiinstances.go:763), and before checkReplicaCreate (:781), the allow-list (:799) and the armed gate (:827).
Every check an explicit region meets, a defaulted one meets too (R2). An explicit home name is non-empty, so it never triggers the fill, and homeRegionIn then turns it into "". That is the override that always exists once §5.6 holds (codex 3).
Each condition earns its place:
| condition | why the default stays out | | --- | --- | | request names a region | an explicit choice wins (R2) | | placed != "" | a fleet placement already governs every instance's region (:785) | | m.Pool != "" | a defaulted region with no box serving the pool mints nowhere | | req.Role != "" | a replica must name its placement; filling it would turn "role replica needs a placement" (:578-581) into a silent acceptance | | not DerivesPaths() | only a socket-backed instance has a process and data that region places | | store read fails | refused, as the placement read above it is (:764-768) |
An app-derived region needs positive eligibility (codex R2-2). The armed gate goes dormant when no file publishes a placement (:848). That dormancy stays for explicit requests and for the legacy home default. For regionFrom == "app" it does not apply. The door reads the edges files strictly (§6.1) and refuses in three cases:
- The read failed: a 503 with C28. The region could not be confirmed, which is different from "absent".
- No eligible box publishes the code: C16, naming
absent,not-hosting,no-secrets-key(R9) ornot-eligible. This covers the last box in a region being retired, and every box losing its placement code. - The allow-list excludes it: C17.
The armed gate's own checks then run as before.
(decision) A failing defaulted region is refused, not quietly replaced by the home. The way out is always an explicit region, the home's name included, and app_default (§9.3) and the Instances tab (§10) show the refusal before anyone tries (codex 7).
(decision) A defaulted home whose region is in its own replicas set is refused (C26). Explicit requests keep today's behaviour.
(decision, R9) An explicit request into a region whose boxes provably lack the deployment's secrets key is refused (C31, 400), for a socket-backed instance (DerivesPaths) and for a replica. Static and addr-backed records need no config on the box and are unaffected.
- What "provably" means. The console's
CARLOS_SECRETS_PUBKEYis set, and every reachable, hosting box publishing the code is a current writer (it publishesaccepts_inbound, which the same box release adds and which is never omitted). Among those boxes, none publishes a matchingsecrets_key_fp. A current writer that publishes no fingerprint holds no key, because the field is omitted exactly then. - Unknown keeps today's behaviour. An old writer, or a console with no
CARLOS_SECRETS_PUBKEY, cannot prove a mismatch, so the request proceeds as it would today. Roll order therefore never refuses a working region. - Why refuse, and not keep today's behaviour. The record this would write is an instance that starts, answers health checks, and runs with no config, no secrets and no store, which is the kpmg outage. It belongs with the existing "reachable but cannot host" refusal (
apiinstances.go:827-852, #110). That one also refuses an explicit request the box would accept but could never serve. A person who named the region learns why at the door, not from a broken app.
Response. apiRecordView (apiinstances.go:535) gains two fields that sit outside the record, like PrivateStoreBucket:
region_from:request,apporfleet, omitted for the home default;region_name(§14).
8. Every create path
8.1 Where an app is created
| path | file:line | change | | --- | --- | --- | | New-app form, GET | handleNewAppForm, server.go:3586; app_new.html | picker with an @auto option (§10), when the switch is on and two or more regions are eligible | | New-app form, POST | handleNewApp, server.go:3607; claim at :3626 | region is @auto or a code; tz from a hidden field; ip from s.clientIP; builds regionReq (§5.5) | | CLI API create | handleAPICreateApp, internal/console/apicli.go:330; body at :359; claim at :415; response at :461 | takes region and timezone; ip from s.clientIP for a person; returns region, region_name, region_source and regions_available | | The claim | claimApp, server.go:3466; insert at :3550 | takes regionReq; resolves once it knows whether this is an adoption; CreateAppInRegion (§5.3) | | carlos apps create | runAppsCreate, apps.go:442; call at :497; print at :502 | --region; sends the timezone; prints the region line (§11) | | carlos deploy onboarding | offerCreateApp, deploy.go:513, reached from runDeploy at :113; call at :536; print at :540 | --region; sends the timezone; prints the region line | | CLI client | apiSession.createApp, apimode.go:994; apiCreateAppResult, :958 | two omitempty fields in, four tolerant fields out | | Legacy owners backfill | BackfillOpsMemberships, store.go:623; insert at :665 | none: the defaults mean the home |
(decision) An explicit region at app create is validated the way the instance door validates one:
ValidRegion.- Armed and hosting, unless the gate is dormant.
- Not provably missing the secrets key (R9, the same rule as §7), refused with C31.
- The home's name is accepted through
homeRegionIn.
Refusals list what would pass (C18, C19, C31). There is no allow-list yet, because there is no manifest yet. A fleet place and a region may both be sent, and both are stored (§12).
8.2 Where an instance is created
| path | file:line | today | change | | --- | --- | --- | --- | | The door | handleAPIInstanceCreate, apiinstances.go:608; region at :772 | "" means the home | the fill and eligibility (§7) | | carlos instances create | instances_cli.go:322 | sends --region or "" | help text; prints region_name (§11) | | carlos canary | canary.go:407 | sends "" | the door fills the app's region; a new --region, with the existing-canary rule (§11) | | carlos deploy static host | mintPlatformHost, deploy.go:858 | static, "" | none | | First-route domain attach | internal/console/domainattach.go:178 | static, no region | none | | External callers (titogo, jam) | POST …/instances, internal/console/authcli.go:613 | usually "" | follow the app's region; can ask for a guess (§9.4) | | Sidecar create | handleAPISidecarCreate, internal/console/apisidecars.go:257 | "" or explicit | none (§12) |
9. API changes
All changes are additive: new request fields are optional and new response fields can be ignored. readJSON is not strict (authcli.go:932-947), so an old console ignores the new request fields instead of refusing them.
9.1 POST /api/cli/apps/{account}/new. The request gains two optional fields:
region: an explicit choice, as a code or the home's name;timezone: a hint, used only when there is noregion, and never stored.
The response gains these fields, echoing what was stored:
{"region": "uk", "region_name": "UK · London", "region_source": "timezone",
"regions_available": 3}regions_availablecounts eligible candidates.- With the switch off, an explicit non-home
regionis answered with the home anddefault, and the CLI says so (C13b).
9.2 POST /api/cli/apps/{account}/{app}/instances. The request is unchanged and behaviour follows §7. The response gains region_from and region_name.
9.3 GET /api/cli/apps/{account}/{app}/regions. The gate, the role (authcli.go:586) and the legacy fields are unchanged (codex 6):
regionsis still the observed topology, unfiltered.defaultis stillHomeRegion.- Each entry gains
aws_regionandname.
The response gains one block:
"app_default": {
"region": "uk", "name": "UK · London", "source": "timezone",
"effective": "uk",
"available": false, "reason": "absent",
"region_known": true, "placement_known": true,
"instance_domain": "bab.oncarlos.com"
}regionis the app's stored region under its name, always. It is the only fielddeploy --regioncompares against (codex R2-4).effectiveis where a create that names no region actually lands: - the stored region; - the home, for an app with no region of its own or a pooled app; - the fleet placement itself (fleet/acme), for a fleet-placed app, never the platform home (codex R2-4).availableandreasondescribeeffective: -absent: the code is reported even thoughregionsdoes not list it (codex 7); -not-hosting; -no-secrets-key(R9, §6.1); -not-eligible(§6.1); -not-allowed; -pooled; -fleet:availableis omitted, because the fleet's own rail decides; -policy-unreadable:availableis omitted; -edges-unreadable:availableis omitted.- An absent manifest is not an error, and no query parameters are added (codex 8).
- (R9a) Two knowledge flags.
/regionsalways answers 200, so these say which facts behindapp_defaultthe console actually read: -region_knownis true exactly when the app's stored region was read. When it is false,region,name,sourceandeffectiveare empty (never the home's name standing in for an unread value), andavailableis omitted. -placement_knownis true exactly when the app's manifest and its fleet placement were read. When it is false,reasonispolicy-unreadableandavailableis omitted. - They separate certainty about the stored region from certainty about the placement policy, because the two CLI checks need different facts. - The R9a emptiness holds whatever the placement says: for a pooled or fleet-placed app whose stored region was not read,effectiveis still empty andavailablestill omitted, andreason(pooled,fleet) is the only placement fact stated (rev 6.2, Astra plan round 4, finding 1). - Absent knowledge fails closed. A client treats a flag that is false, or missing from anapp_defaultblock, as unknown, and refuses the check that needs it.deploy --regionneedsregion_known; refused, it prints C15c.canary --regionneedsplacement_knownand, because its fleet check readseffective,region_knowntoo (rev 6.2); refused, it prints C30b. - Both flags are additive. A console too old to sendapp_defaultat all is handled as before (§9.6, §11): deploy skips its check, and canary refuses--region. - (rev 6.2)
instance_domain. Additive and read-only: the domain the app's instance hosts take, so a client can name an instance's host before writing anything. It is the manifest'sdomainwhen a manifest exists, else the domain the instances-manifest PUT would stamp (<sqid>.<PLATFORM_APPS_DOMAIN>), and it is omitted when the manifest is unreadable or the console has no valid apps domain. - It is the authoritative hostname suffix forcanary --region's preflight: the CLI builds<name>.<app>.<instance_domain>and reads that one record (GET …/instances/{host}) before any write, because the capped instance list skips records it cannot read. A 404 means no canary yet; any other failure, or noinstance_domain, refuses with C30c. - The CLI then checks the instances-manifest PUT's answer against the preflight: the host built from the domain that PUT returns must equal the host it read. If it does not, it stops with C30c before writing any record.
9.4 POST /api/cli/apps/{account}/{app}/regions/guess.
- Same gate and role as §9.3.
- The body is
{"tz": "...", "ip": "..."}, both optional. A person's request with noipuses its own address, and a service principal's never does. - The answer is
{"guess": {"code", "name", "source"}}or{"guess": null}. It isnullfor a pooled or fleet-placed app (§6.1) (codex R4-1). - An unreadable manifest, or an unreadable fleet placement, gets a 502 with C27.
- It writes nothing.
9.5 Browser: POST /a/{sqid}/apps/new/region.
- A form body with
csrfandtz. - Gated by session through
s.account. - Returns
{"code", "name", "source"}. - Writes nothing.
newis a reserved app name (server.go:1150).
9.6 Compatibility and mixed versions.
| client | console | result | | --- | --- | --- | | old CLI | new | No timezone is sent, so IP for a person, then home. Never an error. | | new CLI | old | The response has no region. The CLI prints nothing, or C13a if --region was given. | | new CLI | new, switch off | The app goes to the home. With --region, C13b. | | titogo, unchanged | new | Creates follow the app's region, which for every existing app is the home. default is unchanged. | | any | rolled back below the floor | R7: no-region creates land at the home, and roll-forward lists the possible discrepancies (§5.7). |
10. Console UI
New-app page (app_new.html), when the switch is on and two or more regions are eligible:
- The picker. A
<select name="region">whose first option has the value@autoand the label C3, naming the server's guess (IP, then home). Every eligible region follows, with its code as the value and its place name as the label (C1, C2, C23). A hiddentzfield rides along. @autois not a region (codex 2, codex R2-3). Selecting a code is the person's choice, and leaving@autoasks the server to guess.- The script. An inline progressive-enhancement script, in the style of the account switcher (
templates/layout.html:583), fillstz, POSTs it to §9.5, and relabels only the@autooption. - Otherwise there is no picker, and the form is byte-identical to today's.
- Attribution. DB-IP attribution (C24) shows when the IP step is configured.
POST (handleNewApp):
- An eligible code is
explicit. @auto, or noregionfield at all, makes the server resolve fromtz, then the request's IP, then the home. On an adoption that resolves to"".- Any other value re-renders the form with C5.
- The success flash (
server.go:3644) is existing copy and is untouched. The audit detail (:3643) gains the place (C21).
Instances tab (the Placement section, app_instances.html:60-78), from app_default:
- C8 shows whenever the default cannot accept a create.
- Otherwise C7 shows when the app has a non-home region, or when two or more regions are eligible.
- Pooled and fleet-placed apps show neither line.
11. CLI
Timezone detection is a helper in the root package, with getenv, readlink and readFile injected and only portable calls, so 20-cross builds it everywhere. It tries, in order:
$TZ, with a leading:stripped; a path keeps what followszoneinfo/.readlink /etc/localtime, keeping what followszoneinfo/.- The first line of
/etc/timezone. - Nothing.
On Windows only a valid $TZ is sent. Otherwise the console falls back to IP, then the home.
| command | flag | meaning | | --- | --- | --- | | carlos apps create | --region <code> | The app's region. The default is resolved by the console. | | carlos deploy | --region <code> | The region of the app this deploy creates. On an existing app, it compares against app_default.region (the stored region) (codex R2-4): equal is silent, and different is refused before anything ships (C15). It is skipped against a console with no app_default. A fleet-placed app is judged by its stored region too, because the flag names a region, not a fleet. | | carlos canary | --region <code> | The canary's region. The default is the app's. Before either branch, the CLI reads app_default.effective (§9.3). If the app is fleet-placed, --region is refused before anything is created, rechannelled or uploaded, on the first run and on every repeat (C30) (codex R4-2): the fleet governs every instance's region, so the flag could never be honoured, and refusing it the same way both times keeps the command's answer independent of whether the canary exists yet. Otherwise, for an existing canary host, the record's region is compared with the flag, reading "" as the home's name: equal proceeds, and different is refused before any record write, rechannel or upload (C25) (codex 5). Without --region, a fleet-placed app's canary is created and reused exactly as today. | | carlos instances create | --region <code> | Unchanged. The help says the default is now the app's region. |
Output:
- After "created app …" (
apps.go:502,deploy.go:540), one line per source, C9 to C12, printed only whenregions_available >= 2. instances createandcanaryadd the place to their existing line (C14).- No prompts (R4).
docs/site/cli.md gains the new flags and the new default. There is no new command.
12. Interactions
- Customer fleets.
appPlacement(internal/console/fleets.go:479) overrides the request's region (apiinstances.go:785-797). The precedence is explicit: fleet placement, then the request's region, then the app's region, then the home. A placed app's region is stored, stays dormant, and is reported separately from its placement (§9.3). A placed app gets no instance guess (§6.1), andcanary --regionrefuses on it consistently (§11) (codex R4-1, R4-2). - Pools (
Manifest.Pool). There is no default fill and no guess, andapp_defaultreportspooled. An explicit--regionbehaves as today. - Pool-restricted and no-inbound boxes (codex R2-1) are never eligible for an automatic choice (§6.1). Explicit requests keep today's gate.
- Replicas (#162). A
role: replicacreate is never filled. A defaulted home inside its ownreplicasset is refused (§7). - Moves (#175/#331). A move rewrites one record's
regionat its cut. It never touches the app's region. Steering for moved instances is the follow-on spec's, derived from region (§13.1). - Static. Not filled. Its existing latency default is subject to the §13.3 gate like everything else steered.
- Private storage (#412). Unaffected.
- Sidecars. Unchanged. Their door has no armed gate, so following the app region needs its own design (§20).
- Allow-lists and the home's name.
instanceRegionAllowedcompares the normalized"", so a manifest listing the home's code refuses creates at the home today. This change keeps that and reinterprets no manifest. The README documents it, and rollout audits production manifests first (§17).
13. Latency routing (R6): the requirements Phase L must meet
13.1 Why this is now a separate spec (codex R3-1, R3-2, R3-5)
Rev 3 treated latency routing as two small things: a stamp on each non-home record at mint, and arming the existing converger (#169). Round 3 showed that the converger itself is not ready to arm. Each of these three faults was checked against the code:
- Arming can break DNS for names nobody steered (codex R3-1). The flagship answers every tenant name with one
*.oncarlos.comrecord pointing at the Ireland EIP (carloku-infrastructuredns53.tfanddns.tf; the platform README documents the same single wildcard atREADME.md:442). The converger writes A records at exact names (internal/steering/steering.go:369,:386). One latency record atnew.bap.oncarlos.comcreates the empty non-terminalbap.oncarlos.com. From then on,old.bap.oncarlos.comfindsbapas its closest encloser, looks for*.bap.oncarlos.com, which does not exist, and no longer matches*.oncarlos.com(RFC 4592 §3.3.1). Every unsteered sibling under that account stops resolving. Static platform hosts have been stampedlatencyat mint for weeks (apiinstances.go:936), so arming #169 as the code stands would do this to every account with a static site. It would happen with or without this spec. - Two edges in one AWS region produce an invalid record set (codex R3-2).
desiredRRSetkeys each record by box (SetIdentifier) but stamps it with the edge's AWS region (internal/steering/route53.go:597). Route53 allows one latency record per region within a name and type. A second London box makes every batch fail, on every pass. This spec allows several placement codes in one AWS region (§6.1), so the topology is reachable. - Large withdrawals never complete (codex R3-5). When a pass would delete more than the shrink limit, it refuses every change and returns (
route53.go:207). The loop's own notes say this repeats forever until an operator steps in (internal/host/steering_loop.go:154). Rev 3's claim that deletes "spread over passes" was wrong.
Behind these sits a fourth fact. The flagship's Route53 zone for oncarlos.com is marked "STAGED, NOT YET AUTHORITATIVE" in dns53.tf, and DNSimple still serves the live answers (dns.tf). A converger writing to a zone nobody consults is inert at best.
The arming hazard was posted to amadan discussion carlos/platform #169 on 2026-10-01, so it is on record where the arming decision lives, independent of this spec.
Three rounds have each found a new fault in this one area. Per the lesson this repo has already paid for, the answer is not another patch to an example. It is an invariant, a design built to it, and a simulation that checks it. That is a spec of its own, about the converger. It is not a section of a spec about choosing regions.
(ruling R8) Phase L moves to a follow-on spec, and placement ships without waiting for it. R6 is unchanged: latency routing for non-home instances is still in scope for this work and still binding. It is sequenced, not dropped. This spec now does three things for R6:
- It writes no stamp. Rev 3's mint-time
Steering: "latency"is withdrawn. Every stamped record would join the set of names whose DNS changes when the zone is armed, and that set is exactly the danger above. The follow-on spec should derive "steer this host" from the record's region, not from a stamp: any socket-backed home record with a non-home region, in the shared pool and the platform namespace. That covers instances created before it lands, and instances moved out of the home, with no backfill and no change to the move code. - It states the invariants that spec must meet (§13.2).
- It gates arming #169 on that spec (§13.3).
13.2 Invariants for the follow-on spec
The DNS model is the authoritative zone for the apps domain, the set of armed edges with their AWS regions and health, and the bucket records. Over that model, these must hold after every converge pass, armed or not:
- I1, continuity. Arming, and every later pass, leaves the answer unchanged for every name that has no steered record set of its own. This includes names that sit beneath an empty non-terminal a steered name creates, nested canaries, names whose steering was turned off, and names whose record was deleted. "The answer" is the result of RFC 4592 resolution over the whole zone, not an exact-name lookup.
- I2, validity. For every steered name, the zone holds at most one latency record per AWS region. The record set is a function of the set of eligible edges serving that name, grouped by AWS region. Adding, removing or renaming a box changes the answers inside a region's group. It never changes how many latency records the name has for that region. One way to get this is one latency record per region, aliasing a per-region, health-checked set of that region's edges. The spec chooses.
This groups by region rather than "keyed by edge" because Route53's rule is per region.
- I3, serviceability. Every answer for a steered name is an edge that can serve it: healthy, holding the account's certificate, and able to reach the name's home through the home map. The far edge backhauls; it never answers 404.
- I4, withdrawal. A withdrawal justified by a complete read (a record deleted, steering turned off, a box retired) converges within a bounded number of passes. If it cannot, it raises a named operator alert with a documented procedure. The short-read guard still refuses deletions that a complete read does not justify, but a refusal of deletions never blocks unrelated additions in the same pass.
- I5, dormancy. With no zone configured, nothing is read and nothing is written. This is already true (
steering_loop.go:219). - I6, authority. The converger only writes to the zone the apps domain is actually delegated to. It verifies the delegation before its first write, and stays dormant, loudly, when it cannot.
The proof the follow-on spec must ship: a property test over generated fleets. It covers random edges, random AWS regions including duplicates, random steered and unsteered names including nested and sibling names, and random sequences of creates, deletes, opt-outs, moves and box retirements.
Each scenario runs through desiredState and converge against a Route53 fake. The fake enforces the per-region latency rule, batch atomicity, and an RFC 4592 resolver. The test asserts I1 to I5 after every step, at high seed counts, and each invariant's assertion is checked by mutation: break the code, and watch the test fail. Examples alone have now missed three faults in a row.
13.3 Gates
- Do not arm #169 (
CARLOS_STEERING_ZONE_ID) on any deployment until the follow-on spec has landed and its simulation passes. This holds for the static-site steering that already exists, independently of this spec. - The follow-on spec carries the infrastructure plan for scoped approval (AGENTS.md): - the zone and its delegation; -
steering_zone_idsandsteering_record_names = ["*.<apps-domain>"]on every platform box'sregion-hostmodule (infra/modules/region-host/main.tf:281,variables.tf:153); -CARLOS_STEERING_ZONE_IDin each box'shost.env; - cost, cleanup and disarm. - It also carries the latency measurement. Rev 3's method stands and moves there:
dig +subnetagainst the zone's nameservers, thencurl -w '%{time_connect} %{time_appconnect} %{time_starttransfer}'from a vantage in each region, 30 runs per host, with medians and p90. Take a baseline after this spec's placement ships and before arming, measure again after, and record the numbers in #169.
Until then, a deployment that runs this spec is placement-only. A UK instance is reached through the deployment's static DNS (on the flagship, the Ireland EIP) and backhauled to the UK box. That costs the visitor about one Ireland–London round trip compared with landing in London. It is no worse than today for anyone, and it relieves Ireland's disk for every new non-home instance.
14. Place names
(decision) One table, internal/geo/awsregions.go, maps an AWS region code to {latitude, longitude, place}.
- It covers the AWS commercial regions on AWS's official list as of 2026-10-01 (C23), and records that source and date in a comment.
- Coordinates are the metro area's, each checked by hand when the table is built.
- A region missing from the table shows its bare code and is left out of the guess, with a log line.
- The table is used everywhere: the picker, the Instances tab, every
namein §9, and the CLI lines.
STATUS_BOX_REGION stays, with a different job. It is the public status page's commitment: operator free text, deliberately coarse, and "never derived from cloud metadata" (agent.go:88-92, README :578). It may stay coarser than the picker, but it must agree geographically, and the README says so.
15. Failure modes
| what fails | what happens | | --- | --- | | CONSOLE_GEOIP_DB unset | IP step off: timezone, then home | | mmdb missing, unreadable or corrupt at startup | Logged once; IP step off; the console starts. | | mmdb lookup error | A miss; rate-limited log. | | timezone UTC, Etc/*, unknown or malformed | IP step, then home. | | boxes not yet publishing the eligibility fields | No coded region is eligible. The guess is the home, and app-derived defaults are refused as not-eligible. | | a region's boxes hold no secrets key, or a different one (R9) | Not eligible for any automatic choice. App-derived defaults are refused with C16 no-secrets-key. Explicit socket-backed or replica creates are refused with C31 once a current writer proves the mismatch. | | console has no CARLOS_SECRETS_PUBKEY (R9) | Logged once. No coded region is eligible for automatic placement. Explicit creates keep today's behaviour. | | edges listing fails | The guess is the home. App-derived creates get 503 C28. Explicit creates are unchanged. | | last box of the app's region retired | App-derived creates are refused (C16 absent), and app_default and C8 say why. | | home publishes no AWS region | Every guess is default. Fix it with the box's CARLOS_AWS_REGION. | | an eligible region's AWS region is not in the table | Left out of the guess, still selectable, shown by code. | | app's region excluded by a later allow-list | Refused (C17). | | app-region read fails at the door | 502 (C20); no record written. | | manifest unreadable | The guess endpoint returns 502 (C27). /regions answers policy-unreadable. | | switch on, home unnamed | The switch is treated as off; logged once. | | console binary rolled back below the floor | R7: no-region creates land at the home; roll-forward lists possible discrepancies. | | steering unarmed | Placement-only, with no error. A non-home instance is reached through the static DNS and backhauled (§13.3). | | a manifest's allow-list excludes the home, and no hint maps | "guess": null, never the home (§6.4). | | owner adopts an existing app | ""/default, unless region is explicit. |
16. Test plan
internal/geo.
- Haversine on known pairs: Dublin to London is about 464 km, within 5.
- Nearest: - a single candidate, and a clear winner; - ties go to the home, then the smaller code; - two codes in one AWS region; - an ineligible candidate nearer than every eligible one is never picked; - a candidate with no point is excluded, a home with no point gives
default, and no candidates givesdefault. - Timezones: -
Europe/LondonandEurope/Amsterdammap; -Asia/CalcuttaandAsia/Kolkatagive one point; -America/Argentina/Buenos_Airesmaps, and matching is case-insensitive; -UTC,Etc/UTC,Etc/GMT+5,GMT,Factory,Local,"", a 65-byte name,../etc/passwdand a link loop are all unmappable. - Country points: every multi-row country has an override.
- mmdb, against pinned fixtures: - unset, missing, truncated and non-mmdb files; - City and Country-only records; - private, loopback and CGNAT addresses never reach the reader.
Secrets key (R9):
appconfig.KeyFingerprintof a key's public half equals the fingerprint computed from the base64CARLOS_SECRETS_PUBKEYof the same key.- A box with the key publishes
secrets_key_fp. With no key, or a key that does not parse, it omits the field. The body never contains the private key or the public key itself. - Eligibility: - a box with a matching fingerprint is eligible; - a box with a different fingerprint, or none, is
no-secrets-key; - a console with noCARLOS_SECRETS_PUBKEYmakes no coded region eligible. - An explicit socket-backed create into a region where only current writers publish a mismatch is refused with C31. Under an old writer, or with no console key, it is accepted as today. Static and addr-backed creates are never refused for it.
Edges file and eligibility (codex R2-1):
- The writer publishes
serves_shared_poolandaccepts_inboundfromCARLOS_POOLSandCARLOS_INBOUND, for: - unset; -CARLOS_POOLS=tito; -CARLOS_POOLS=,tito, which the parser reads as["tito"], so it publishesserves_shared_pool: false(codex R3-3); -CARLOS_INBOUND=none. - The published fields are asserted equal to
poolServedandcfg.Inbound != InboundNoneover the sameprovisionTickConfigoutput, for a table ofCARLOS_POOLSandCARLOS_INBOUNDspellings. That makes publication and reconciliation one predicate. - The reader treats absent as unknown and false as no.
- Eligibility needs all three fields true on one reachable box.
- The guess against a named-pool-only box returns no guess for that code.
- The guess against a no-inbound box does the same.
- In both cases, the matching reconcile pass on that box refuses the record (
reconcile.go:925,:1339). That pins why the guess must not go there.
Store.
- Four database states: fresh; from before the columns; from before
migrateAppsKeybut carrying the new ALTERs; andmigraterun twice. - Pre-existing rows read
""/"". CreateAppInRegionwrites both columns in one statement.- Rename and transfer keep the region.
The door (apiinstances_test.go):
- No region, with app region
ukeligible: the record saysukandregion_fromsaysapp. - App region
"": the record is byte-identical to the pre-change golden. - Explicit wins over the app, and fleet wins over both.
- An explicit home name gives
"". - Pooled, static, addr-backed and replica creates are not filled, and the replica refusal is word for word.
- An unavailable app region gives C16, the allow-list C17, a store error 502, and a defaulted home inside its
replicasC26. - Dormancy versus app-derived defaults (codex R2-2): - With no placement published, an explicit
ukis accepted, as today. - An app-derivedukis refused with C16absent. - The last UK box's file removed: an app-derivedukis refused. - The edges listing fails: an app-derived create gets 503 C28, and an explicit create is unchanged. - The switch off with an existing
ukrow: the fill still applies.
No stamp (§13.1). A non-home instance created by this spec carries no Steering field. The record is byte-identical to one created by instances create --region before this spec.
The guess invariant (codex R3-4). A property test over generated policies, candidate sets, missing coordinates, unmappable hints and edges-read failures asserts two things:
- every non-null guess passes
instanceRegionAllowed(m.Regions, homeRegionIn(code)); - a non-null guess exists whenever some filtered candidate is reachable by the order in §6.4.
Named cases:
- an allow-list of only
uk, with the home's coordinates missing; - an unmappable hint against the same list;
- an edges-read failure against the same list, which gives
null.
Pools, end to end. A pooled manifest gives guess: null and app_default.reason: pooled.
Fleets, end to end (codex R4-1). For an app placed on fleet/acme with an unrestricted shared-pool manifest:
/regions/guessreturnsnull;- a create that submits
ukanyway produces a record whose region isfleet/acme, which pins why the guess is withheld; - a failed placement read gives 502 C27.
Service through the real path (codex R4-3), as an integration test with two edges and a backhaul between them:
- a non-home instance answers a request that enters through the other edge;
- with no mesh key mirrored, the same request gets the unknown-host answer (
internal/edge/edge.go:2699,:1428), and the Phase P gate (§17) treats that as a failure, not a pass.
App create.
- An old body from a person resolves by IP, then home. This needs
ippassed throughregionReq(codex R2-5). - A
timezonebody. - Explicit regions: valid, invalid (C18) and unarmed (C19).
placeplusregion.- The switch off, or on with an unnamed home.
- A service principal gets no IP step.
- Adoption: - with
@auto, or atimezonebut noregion, it stores""; - with an explicit code, it stores the code; - a region namedauto, and a home namedauto, both work and are distinguishable from@auto(codex R2-3).
Browser.
- One region, or the switch off: the form is byte for byte today's.
@autois labelled with the IP guess.- §9.5 needs a session and CSRF.
- An explicit code records
chosen.
Endpoints.
/regionsfor an existing app matches today's golden inregionsanddefault.app_defaultcovers eachreason.- For an app storing
ukand placed onfleet/acme:regionisuk,effectiveisfleet/acme, anddeploy --region ukpasses whiledeploy --region ieis refused (codex R2-4).
Rollback list (codex R2-6). A non-home app is not listed for any of:
- its static apex;
- an addr-backed record;
- a replica;
- a pooled record.
A no-region instance record created after the app is listed as a possible discrepancy, worded as such.
Privacy. With a canonical-host redirect configured, a guess POST's tz and ip appear in none of:
- the
Locationheader; - the redirect body;
- any error body;
- the captured logs;
- any audit row.
CLI.
- Timezone detection with injected functions.
- One output line per source.
- C13a and C13b.
deploy --region: different is refused before anything ships, and equal is silent.canary --regionagainst an existing canary in another region is refused before any write or upload.canary --region ukon a fleet-placed app, run twice: both runs refuse with C30 before any create, rechannel or upload. The first run creates nothing, so the second sees the same state (codex R4-2).- Tests use
carlosTestEnvwith a scratchcmd.Dir. 20-crosscovers the CLI changes.
17. Rollout
Phase P: placement.
- Before any roll. - Audit production manifests for
regions(§12). - Confirm each box publishes its AWS region. In particular, confirm the flagship's Ireland box publishesregion: eu-west-1and noplacement. UseCARLOS_AWS_REGIONwhere it does not. - SetCONSOLE_HOME_REGIONon every console that will turn the switch on. - Stage the secrets key (R9). Every box in a region that should take placed instances gets the deployment's
CARLOS_SECRETS_KEYin/etc/carlos/host.env, from the deployment's secret store. On the flagship today that means the UK, US and AU boxes. This is infrastructure work in carloku-infrastructure, under the scoped-approval rule (AGENTS.md). SetCARLOS_SECRETS_PUBKEYon the console if it is not already set. - Box release. It carries the edges-file eligibility fields and
secrets_key_fp. Roll it to every box. Then check that every region's files carry all of them, and that eachsecrets_key_fpequals the console's. Until then, no coded region is eligible (§6.1). - Console release, switch off. It carries the migration,
internal/geo, the fill, the endpoints and the UI. Behaviour is unchanged, and it can be rolled back freely. - Verify on dev. Turn the switch on with two eligible regions. Then: - create an app in the browser, and read its row; - create an instance with no region, and read its record; - diff
/regionsfor an old app against the pre-roll answer; - run the service gate below. - Service gate on the flagship, with the switch still off (codex R4-3).
Eligibility proves configuration, not a working request path. Reachability is only a non-empty EIP (instancesplacement.go:185), and backhaul refuses to dial without a mirrored mesh key (edge.go:2699). So before the switch goes on, every eligible region must serve a fresh instance through the real public path.
For each eligible region R, create one probe instance of a test app with an explicit --region R. That is ordinary API use and needs the switch off. Deploy a known build to it, then check:
- From outside, through the platform host: curl -s -D - -o /dev/null https://<probe host>/ from a machine outside the deployment, using public DNS as it stands. On the flagship that enters through the Ireland edge, so every non-home region is exercised over backhaul. Expect 2xx, and an X-Carlos-Version equal to the deployed build. - Through an edge that is not the instance's box: for every armed edge E other than R's box, curl -s -D - -o /dev/null --resolve <probe host>:443:<E's EIP> https://<probe host>/, with the same expectations. This covers the home region too, entered from a far edge, and every edge-to-box pair the static DNS could someday route.
A 404 with the unknown-host answer, a TLS failure or a version mismatch fails the gate. The switch is turned on only when every eligible region passes. A region that fails is fixed first (mesh key, certificate, activator), or its boxes are made ineligible, before the switch goes on. The probe instances are deleted afterwards. The same gate is re-run before the switch is turned on again after any period off, and whenever a new region becomes eligible.
- Flagship. Roll the console, then turn the switch on. From here the binary rollback floor applies (R7). That release's notes and the roll recipe in carloku-infrastructure name the floor version. Optionally, set
CONSOLE_GEOIP_DBto a DB-IP Lite file. - CLI release. It is safe in either order (§9.6).
- titogo, in its own repo, adopts
/regions/guessand the place names.
No step of Phase P depends on Phase L. Phase P writes nothing to DNS, stamps no Steering field, and needs neither CARLOS_STEERING_ZONE_ID nor any change to the zone.
Phase L: latency (R6, R8) is the follow-on spec's rollout, not this one's (§13.3). Its gate is that spec landing with its simulation passing, plus scoped approval for the infrastructure. Until then, no deployment arms CARLOS_STEERING_ZONE_ID, including for the static-site steering that already exists. The latency baseline is taken once Phase P step 7 has run. That makes Phase L depend on Phase P, never the other way round.
Copy review (§19) happens before Phase P step 4 ships any string.
18. Questions for Paul
None. R8 settled rev 4's sequencing question.
19. Copy (pending copy review)
These are drafts, and none ships until Paul has reviewed it through copy review. There are no em dashes. {…} is filled at runtime.
| id | where | draft | | --- | --- | --- | | C1 | new-app page, picker label | Region | | C2 | new-app page, help | Where this app's instances run. We picked the region nearest to you. Choose another if most of your users are somewhere else. | | C3 | new-app page, @auto option | {place} (nearest to you) | | C5 | new-app page, flash | That region can't take new apps right now. Choose another. | | C7 | Instances tab | New instances start in {place}, this app's region. | | C8 | Instances tab, default unavailable | New instances can't start in {place}, this app's region, right now. Pass a region to place one elsewhere. | | C9 | CLI, from timezone | Region: {place} (from your timezone). Use --region to choose another. | | C10 | CLI, from IP | Region: {place} (from your network location). Use --region to choose another. | | C11 | CLI, fallback | Region: {place} (the platform default). Use --region to choose another. | | C12 | CLI, chosen | Region: {place}. | | C13a | CLI, old console ignored --region | This console does not record an app region yet, so --region was not applied. | | C13b | CLI, console did not apply --region | This console is not assigning app regions yet, so --region was not applied. This app's instances start in {place}. | | C14 | CLI, instances create and canary | created instance {host} (channel {channel}, {place}) | | C15 | CLI, deploy --region on an existing app | {app} already runs in {place}. --region only applies when deploy creates an app. To place one instance elsewhere, use carlos instances create --region. | | C16 | API, defaulted region unavailable | this app's region {code} can't take new instances right now ({reason}). Pass a region to place this instance elsewhere. Regions that can: {list}. | | C17 | API, defaulted region not allowed | this app's region {code} is not one its instances may use. Allowed: {list}. Pass a region to choose one. | | C18 | API, app create, unknown region | region {code} is not one this platform runs in. Choose from: {list}. | | C19 | API, app create, region not armed | region {code} can't take new apps right now. Choose from: {list}. | | C20 | API, app region unreadable | could not read this app's region. Try again. | | C21 | activity feed | created app {name} in {place} | | C22 | flag help | apps create: "region this app's instances start in (default: the region nearest you)". deploy: "region for the app this deploy creates (default: the region nearest you)". canary: "region for the canary instance (default: the app's region)". instances create: "region for this instance (default: the app's region, subject to the manifest's allow-list)". | | C24 | new-app page, IP step configured | IP location by DB-IP. (Linked to db-ip.com; DB-IP's required wording to be confirmed against its licence page.) | | C30 | CLI, canary --region on a fleet-placed app | {app} runs on your fleet {fleet}, which decides where every instance runs, so --region does not apply. Run the command without --region. | | C31 | API, explicit create into a region without the secrets key (R9) | region {code} can't run instances right now: its boxes don't hold this deployment's secrets key, so an instance there would start without its config or secrets. Choose another region. | | C25 | CLI, canary --region on an existing canary | canary {host} already runs in {place}. --region applies only when canary creates the host. Run carlos canary remove first to start it somewhere else. | | C26 | API, defaulted home inside its replica set | this app's region {code} is also one of this instance's replica placements. Pass a region for the instance, or drop {code} from replicas. | | C27 | API, guess with unreadable policy | could not read this app's instance policy, so there is no region suggestion. Try again. | | C28 | API, app region unconfirmed | could not confirm that this app's region {code} can take new instances. Try again, or pass a region. | | C29 | roll-forward log, read by operators | possible placement discrepancy: {host} of {app} has no region, but {app}'s region is {code}. It may be a deliberate home override; review it. |
The {reason} words in C16 are absent, not-hosting, no-secrets-key (R9) and not-eligible. Shown to a person, they read: "no box runs there now", "boxes there can't start instances", "boxes there can't open this deployment's secrets" (R9), and "boxes there don't take ordinary instances". Those are copy too.
C23, place names:
| AWS region | place | AWS region | place | | --- | --- | --- | --- | | us-east-1 | US East · N. Virginia | eu-north-1 | Sweden · Stockholm | | us-east-2 | US East · Ohio | il-central-1 | Israel · Tel Aviv | | us-west-1 | US West · N. California | me-south-1 | Bahrain | | us-west-2 | US West · Oregon | me-central-1 | UAE | | ca-central-1 | Canada · Montréal | af-south-1 | South Africa · Cape Town | | ca-west-1 | Canada · Calgary | ap-south-1 | India · Mumbai | | mx-central-1 | Mexico · Querétaro | ap-south-2 | India · Hyderabad | | sa-east-1 | Brazil · São Paulo | ap-east-1 | Hong Kong | | eu-west-1 | Ireland · Dublin | ap-east-2 | Taiwan · Taipei | | eu-west-2 | UK · London | ap-southeast-1 | Singapore | | eu-west-3 | France · Paris | ap-southeast-2 | Australia · Sydney | | eu-central-1 | Germany · Frankfurt | ap-southeast-3 | Indonesia · Jakarta | | eu-central-2 | Switzerland · Zurich | ap-southeast-4 | Australia · Melbourne | | eu-south-1 | Italy · Milan | ap-southeast-5 | Malaysia | | eu-south-2 | Spain · Aragón | ap-southeast-6 | New Zealand · Auckland | | ap-northeast-1 | Japan · Tokyo | ap-southeast-7 | Thailand · Bangkok | | ap-northeast-2 | South Korea · Seoul | ap-northeast-3 | Japan · Osaka |
The docs/site/cli.md, README and release-note additions (§5.7, §11, §12, §14) are prose, and go through the same review.
20. Follow-ups (not in this spec)
- Changing an existing app's region (R5). An owner-only verb that affects future creates only and leaves existing instances alone.
- Phase L: latency routing (R6), on its own spec, meeting §13.2's invariants with a simulation over generated fleets. It derives "steer this host" from the record's region, which also covers moved instances. It carries #169's infrastructure plan, the delegation check for the flagship's
oncarlos.comzone, and the latency measurement. - Pool-aware candidates. Pool eligibility proven by the target box itself.
- Sidecars following the app region, with an armed gate on their door.
- A browser instance-create form.
- Tightening #162 for explicit home-in-own-replica-set creates.
21. Review dispositions
Round 1 (rev 1, 7fbc9479)
| # | sev | finding | disposition | | --- | --- | --- | --- | | 1 | P1 | A picker submitting the guess bypasses the pool exclusion | accepted. No guess for a pooled policy (§6.1). | | 2 | P1 | Browser adoption silently changes an existing app's default | accepted. regionReq with guess resolution after adoption (§5.5). | | 3 | P1 | An unnamed home makes override and recovery impossible | accepted. Non-home writes need a named home (§5.6). | | 4 | P1 | No mixed-version or rollback rule | accepted. Read side first, the switch as lever, the binary floor (§5.7). Round 2 marked it "partly" pending acceptance of the floor; R7 settles that. | | 5 | P2 | canary --region undefined for an existing canary | accepted (§11). | | 6 | P2 | Manifest absence and read failure unspecified | accepted. Three policy states (§6.1, §9.3). | | 7 | P2 | Unavailable-default explanation disappears | accepted. app_default (§9.3), C8 (§10). | | 8 | P2 | Query hints reflected by redirects | accepted. POST bodies only (§6.5, §9.4, §9.5). |
Round 2 (rev 2, bd4ed0c4)
| # | sev | finding | disposition | | --- | --- | --- | --- | | R2-1 | P1 | Automatic candidates can include pool-restricted or no-inbound boxes that never mint shared instances | accepted. Two additive edges-file fields, positive eligibility for automatic choices, and a box release before the switch (§6.1, §17). I chose publication over a console-side operator declaration because the box is the only party that knows its own pools and inbound setting, and the hibernation_configured precedent is already there. | | R2-2 | P1 | Losing the last placement publication disables the unavailable-default refusal | accepted. App-derived defaults need positive eligibility and never ride dormancy. A read failure is distinct (503 C28). The contradicting test is replaced (§7, §16). | | R2-3 | P2 | auto collides with a valid placement code | accepted. The sentinel is @auto, outside the grammar, with tests for a region and a home named auto (§5.5, §10). | | R2-4 | P2 | Fleet app_default.code misleads the deploy preflight | accepted. app_default.region (stored) is separate from effective (the fleet itself), and deploy compares against region (§9.3, §11). | | R2-5 | P2 | regionReq cannot carry the IP claimApp needs | accepted. A request-local ip from s.clientIP in regionReq, resolved after adoption, never logged or stored (§5.5). | | R2-6 | P2 | The rollback detector flags correct deployments | accepted. Non-inheriting shapes are excluded, matches are worded as possible discrepancies for review, and the runbook records the interval (§5.7, C29). |
Astra's recommendations on rev 1's open questions
| rev 1 Q | disposition | | --- | --- | | Q1 titogo adoption in "done" | superseded by R5: done is new apps plus moves | | Q2 placement-only, measure first | superseded by R6: latency is in scope. Measurement moves to the follow-on spec (§13.3). | | Q3 refuse an unavailable default | adopted (§7) | | Q4 creation-only region | adopted; the change verb is a follow-up (R5, §20) | | Q5 defer the browser instance form | adopted (§20) | | Q6 DB-IP Lite, corrected GeoLite2 note | adopted (§6.2, C24) | | Q7 pinned reader and fixtures | adopted (§6.2) | | Q8 picker style, geographic consistency | adopted (§14) | | Q9 verify home publication, CARLOS_AWS_REGION | adopted (§17) | | Q10 sidecars unchanged | adopted (§20) | | Q11 suppress pooled guesses | adopted (§6.1) | | Q12 home inside own replica set | adopted for defaulted regions (§7); explicit is a follow-up (§20) | | Q13 keep allow-list normalization, audit | adopted (§12, §17) | | Q14 unnamed home | adopted, as finding 3 | | Q15 Windows fallback | adopted (§11) | | Q16 AWS list | adopted (§14) |
Astra's round-2 recommendations on rev 2's questions
| rev 2 Q | disposition | | --- | --- | | Q1 (b): titogo in "done" | superseded by R5 | | Q2 (a): placement-only, measure first | superseded by R6. Measurement moves with Phase L to the follow-on spec (§13.3). | | Q3 (a): accept the floor | matches R7 |
Round 3 (rev 3, 8e59f598)
| # | sev | finding | disposition | | --- | --- | --- | --- | | R3-1 | P1 | Arming per-host steering breaks DNS for unsteered siblings, through empty non-terminals shadowing *.<apps-domain> | accepted. Verified against steering.go:369, :386 and carloku's single wildcard (dns53.tf, dns.tf). It already affects static-site steering. Resolved by invariant I1, the gate on arming #169, and the move of Phase L to a follow-on spec (§13). | | R3-2 | P1 | One latency record per edge is invalid when two edges share an AWS region | accepted. Verified at route53.go:597. Resolved by invariant I2, which groups by AWS region, and a Route53 fake that enforces the rule (§13.2). This departs from the coordinator's example wording "keyed by edge, not region", because Route53's constraint is per region. | | R3-3 | P1 | The CARLOS_POOLS=,tito test would publish shared eligibility the reconciler denies | accepted. Verified at agent.go:2033. Publication is computed from the same ReconcileConfig and predicate, ,tito publishes false, and a test pins the equality (§6.1, §16). | | R3-4 | P2 | Home fallback is not constrained by the allow-list | accepted. The guess invariant, with the home fallback only when the home passes the policy, and a property test (§6.4, §16). | | R3-5 | P2 | Large DNS withdrawals are refused indefinitely, not spread over passes | accepted. Verified at route53.go:207 and steering_loop.go:154. The wrong claim is removed, and invariant I4 makes it the follow-on spec's requirement (§13). | | R3-6 | P3 | §4 reverses which requests need positive eligibility | accepted. Corrected to app-derived (§4). |
Astra's round-3 recommendations
| former question | disposition | | --- | --- | | Q1: apply R5, titogo separate | adopted (§2, §17 step 9) | | Q2: implement R6 with the DNS corrections, keep the separate approved Phase L rollout and measurements | adopted in substance. The corrections are larger than a patch, so they move to the follow-on spec with invariants (§13); the sequencing is ruled by R8. | | Q3: name the exact floor version before activation; keep region-aware reads when disabling | adopted (§5.6, §5.7, §17 step 7) |
Round 4 (rev 4.1, 4ad64f7c)
| # | sev | finding | disposition | | --- | --- | --- | --- | | R4-1 | P2 | Fleet-placed apps still get instance guesses the door will override | accepted. Verified at fleets.go:479 and apiinstances.go:798. The guess endpoint resolves fleet placement first and returns null; app-creation inference is unchanged; end-to-end test (§6.1, §9.4, §12, §16). | | R4-2 | P2 | canary --region on a fleet behaves differently before and after creation | accepted. Verified at canary.go:405 and apiinstances.go:798. Fleet precedence is resolved before either branch, and --region on a fleet-placed app refuses identically every time with C30, never suggesting a remove (§11, §12, §16). | | R4-3 | P2 | Phase P verification can pass while non-home instances are unreachable | accepted. Verified at instancesplacement.go:185, edge.go:2699 and :1428. A service gate before the production switch: a fresh probe per eligible region, reached from outside through the platform host and through every other edge with --resolve, checking 2xx and X-Carlos-Version. All regions must pass. Plus an integration test for the missing-mesh-key case (§16, §17 step 6). |
Astra raised no open questions in round 4. §18 stands at none.
Amendment rev 6.2 (instance_domain, Paul 2026-10-03)
Not a review finding of this spec. Astra's plan round 3 (finding 1) showed that canary --region, deciding from the capped instance list, could miss an existing canary and write. The plan answered with instance_domain and a single-record read. Astra's plan round 4 judged it sound, including the check of the PUT's answer against the preflight, and Paul approved it. §9.3 carries it, with the round-4 consequence of R9a for pooled and fleet apps.
Amendment R9a (rev 6.1, Paul 2026-10-02)
Not a review finding of this spec. Astra's review of the implementation plan (plan round 2, finding 3) showed that a 200 marked policy-unreadable let the CLI read an unread region as the home. Paul accepted the two flags that fix it. Astra's plan round 3 judged them sound. §3 (R9a) and §9.3 carry them, and the copy record gains C15c for deploy's refusal.
Amendment R9 (rev 6, Paul 2026-10-02)
Not a review finding: a production finding, ruled by Paul. Recorded here so the dispositions list every change since rev 5. The changes are §3 (R9), §6.1 (secrets_key_fp, its format, the console's expected fingerprint, the fourth eligibility fact), §7 and §8.1 (the explicit refusal, C31), §9.3 and C16 (the no-secrets-key reason), §15, §16, §17 (staging the key), and §19 (C31).