carlos / platform Public

Merge remote-tracking branch 'origin/main' into design-sidecar-isolation

claude agent, operated by paul@keymail.dev 4abca90ad7802e635dc3cdff2b543d0ac3b3165c parent ca0b439parent e3fd2e0
29 files changed, +1779 −63
  • AGENTS.md +44 −0
  • Makefile +22 −1
  • README.md +7 −0
  • cmd/console/main.go +54 −0
  • cmd/console/sidecaraccounts_serve_test.go +55 −0
  • cmd/console/sidecaraccounts_test.go +44 −0
  • docs/site/cli.md +6 −0
  • docs/superpowers/notes/census-proposal-console-brand-name.md +13 −0
  • docs/superpowers/notes/census-proposal-email-signin.md +4 −2
  • docs/superpowers/notes/census-proposal-sec-deps-dos-bumps.md +37 −0
  • docs/superpowers/notes/census-proposal-sec-host-imdsv2-module.md +28 −0
  • docs/superpowers/notes/census-proposal-sec-sidecar-create-gate.md +13 −0
  • go.mod +11 −12
  • go.sum +22 −34
  • infra/modules/region-host/main.tf +13 −0
  • infra/modules/region-host/tests/instance_metadata.tftest.hcl +49 −0
  • internal/console/apisidecars.go +13 −1
  • internal/console/server.go +7 −0
  • internal/console/sidecarsgate.go +64 −0
  • internal/console/sidecarsgate_test.go +193 −0
  • internal/provision/census_propose_internal_test.go +373 −0
  • internal/provision/census_propose_test.go +571 −0
  • internal/provision/plan14_root_reads_test.go +88 −12
  • internal/provision/plan9_b8_census_test.go +1 −1
  • internal/provision/sec_sidecar_create_gate_census_test.go +27 −0
  • internal/provision/testdata/plan14-root-reads-chain.tsv +6 −0
  • internal/provision/testdata/plan14-root-reads-sec-deps-dos-bumps.tsv +3 −0
  • internal/provision/testdata/plan14-root-reads-sec-host-imdsv2-module.tsv +5 −0
  • internal/provision/testdata/plan14-root-reads-sec-sidecar-create-gate.tsv +6 −0

CI passed — run details

CI log
=== gate ===
.amadan/test-gate.sh
go: downloading golang.org/x/net v0.60.0
go: downloading golang.org/x/crypto v0.57.0
go: downloading golang.org/x/sys v0.48.0
go: downloading golang.org/x/sync v0.23.0
go: downloading github.com/pion/stun/v3 v3.1.5
go: downloading github.com/pion/transport/v4 v4.0.2
go: downloading golang.org/x/text v0.42.0
go: downloading golang.org/x/text v0.42.0
go: downloading github.com/pion/dtls/v3 v3.1.4
go: downloading github.com/pion/dtls/v3 v3.1.4
=== launchctx (0 s)
ok  	amadan.net/carlos/platform.git/launchctx	0.013s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
=== vet (25 s)
go: downloading golang.org/x/tools v0.49.0
go: downloading golang.org/x/text v0.42.0
go: downloading github.com/pion/dtls/v3 v3.1.4
go: downloading golang.org/x/mod v0.41.0
=== gofmt (1 s)
gofmt -l .: clean
=== rest (186 s)
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.663s
ok  	github.com/carlosframework/platform/cmd/moveprobe	11.858s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.781s
ok  	github.com/carlosframework/platform/internal/activator	59.177s
ok  	github.com/carlosframework/platform/internal/agentsock	0.029s
ok  	github.com/carlosframework/platform/internal/alerting	0.010s
ok  	github.com/carlosframework/platform/internal/analytics	0.046s
ok  	github.com/carlosframework/platform/internal/appconfig	2.938s
ok  	github.com/carlosframework/platform/internal/apperr	0.145s
ok  	github.com/carlosframework/platform/internal/applog	0.240s
ok  	github.com/carlosframework/platform/internal/awsinv	0.007s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.007s
ok  	github.com/carlosframework/platform/internal/boxid	0.657s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.017s
ok  	github.com/carlosframework/platform/internal/caps	0.020s
ok  	github.com/carlosframework/platform/internal/certwarm	0.007s
ok  	github.com/carlosframework/platform/internal/cgroup	0.849s
ok  	github.com/carlosframework/platform/internal/changelog	0.569s
ok  	github.com/carlosframework/platform/internal/claimactivation	1.258s
ok  	github.com/carlosframework/platform/internal/clamd	0.064s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	7.925s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.014s
ok  	github.com/carlosframework/platform/internal/dbquery	0.008s
ok  	github.com/carlosframework/platform/internal/dbreads	2.139s
?   	github.com/carlosframework/platform/internal/dbreads/dbreadstest	[no test files]
ok  	github.com/carlosframework/platform/internal/dbverify	0.007s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.012s
ok  	github.com/carlosframework/platform/internal/docsite	0.018s
ok  	github.com/carlosframework/platform/internal/domains	2.827s
ok  	github.com/carlosframework/platform/internal/doorkey	0.007s
ok  	github.com/carlosframework/platform/internal/economics	0.010s
ok  	github.com/carlosframework/platform/internal/edge	49.088s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.009s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.152s
ok  	github.com/carlosframework/platform/internal/gate	1.667s
ok  	github.com/carlosframework/platform/internal/genselect	0.018s
ok  	github.com/carlosframework/platform/internal/geo	0.020s
ok  	github.com/carlosframework/platform/internal/geoip	0.032s
ok  	github.com/carlosframework/platform/internal/historyreadback	0.218s
ok  	github.com/carlosframework/platform/internal/host	150.524s
ok  	github.com/carlosframework/platform/internal/hostactors	6.539s
ok  	github.com/carlosframework/platform/internal/hostactors/awsmembership	0.017s
ok  	github.com/carlosframework/platform/internal/hostbox	3.049s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.760s
ok  	github.com/carlosframework/platform/internal/hostenv	0.020s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.017s
ok  	github.com/carlosframework/platform/internal/hostexec	5.423s
ok  	github.com/carlosframework/platform/internal/hostfence	0.056s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.471s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.017s
ok  	github.com/carlosframework/platform/internal/hostguard	1.073s
ok  	github.com/carlosframework/platform/internal/hostops	1.973s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.020s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.017s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.629s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.183s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.017s
ok  	github.com/carlosframework/platform/internal/infraview	0.017s
ok  	github.com/carlosframework/platform/internal/instancectl	5.259s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.857s
ok  	github.com/carlosframework/platform/internal/instancefs	0.021s
ok  	github.com/carlosframework/platform/internal/instancekey	0.018s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.008s
ok  	github.com/carlosframework/platform/internal/layout	1.788s
ok  	github.com/carlosframework/platform/internal/lbhealth	0.019s
ok  	github.com/carlosframework/platform/internal/lease	10.315s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.632s
ok  	github.com/carlosframework/platform/internal/ledger	0.028s
ok  	github.com/carlosframework/platform/internal/mail	0.279s
ok  	github.com/carlosframework/platform/internal/maintenance	0.103s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.573s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.118s
ok  	github.com/carlosframework/platform/internal/metrics	0.011s
ok  	github.com/carlosframework/platform/internal/moves	29.676s
ok  	github.com/carlosframework/platform/internal/moves/archive	1.028s
ok  	github.com/carlosframework/platform/internal/objstore	0.320s
ok  	github.com/carlosframework/platform/internal/ops	0.219s
ok  	github.com/carlosframework/platform/internal/park	0.280s
ok  	github.com/carlosframework/platform/internal/plans	0.012s
ok  	github.com/carlosframework/platform/internal/privatefile	0.009s
ok  	github.com/carlosframework/platform/internal/provision	51.455s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.012s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	5.687s
ok  	github.com/carlosframework/platform/internal/relay	7.243s
ok  	github.com/carlosframework/platform/internal/release	32.965s
ok  	github.com/carlosframework/platform/internal/rollout	3.882s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.004s
ok  	github.com/carlosframework/platform/internal/rootaws	0.008s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.062s
ok  	github.com/carlosframework/platform/internal/rootstate	0.036s
ok  	github.com/carlosframework/platform/internal/runs	0.618s
ok  	github.com/carlosframework/platform/internal/s3compat	0.008s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.652s
ok  	github.com/carlosframework/platform/internal/seeds	0.013s
ok  	github.com/carlosframework/platform/internal/sim/engines	2.567s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	13.415s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.699s
ok  	github.com/carlosframework/platform/internal/status	0.095s
ok  	github.com/carlosframework/platform/internal/steering	0.009s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.322s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.007s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.007s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.005s
ok  	github.com/carlosframework/platform/internal/usage	0.006s
ok  	github.com/carlosframework/platform/internal/version	0.004s
ok  	github.com/carlosframework/platform/internal/vet	1.690s
=== internal_console@0 (82 s)
ok  	github.com/carlosframework/platform/internal/console	80.173s
=== internal_console@1 (49 s)
ok  	github.com/carlosframework/platform/internal/console	47.521s
=== internal_console@2 (46 s)
ok  	github.com/carlosframework/platform/internal/console	44.511s
=== internal_console@3 (50 s)
ok  	github.com/carlosframework/platform/internal/console	47.878s
=== internal_console@4 (47 s)
ok  	github.com/carlosframework/platform/internal/console	45.457s
=== internal_console@5 (48 s)
ok  	github.com/carlosframework/platform/internal/console	46.032s
=== internal_console@6 (44 s)
ok  	github.com/carlosframework/platform/internal/console	41.760s
=== internal_console@7 (51 s)
ok  	github.com/carlosframework/platform/internal/console	49.321s
=== root@0 (114 s)
ok  	github.com/carlosframework/platform	111.893s
=== root@1 (85 s)
ok  	github.com/carlosframework/platform	83.192s
=== root@2 (97 s)
ok  	github.com/carlosframework/platform	94.846s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 50659490 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: cached, digest verified
  mc: cached, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.024s
=== RUN   TestT4StorageProbeMinio
2026/10/24 20:38:18 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18E191ACDF304194, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.022s
=== RUN   TestS3StoreMinioContract
=== RUN   TestS3StoreMinioContract/conditional_get_and_etag_listing
=== RUN   TestS3StoreMinioContract/bump_through_every_mutation_path
--- PASS: TestS3StoreMinioContract (0.03s)
    --- PASS: TestS3StoreMinioContract/conditional_get_and_etag_listing (0.01s)
    --- PASS: TestS3StoreMinioContract/bump_through_every_mutation_path (0.02s)
PASS
ok  	github.com/carlosframework/platform/internal/release	0.032s
Bucket created successfully `local/dbf-deploy`.
Bucket created successfully `local/dbf-data-ie`.
Bucket created successfully `local/dbf-data-uk`.
local/dbf-deploy versioning is enabled
Created policy `dbf-console` successfully.
Created policy `dbf-ukbox` successfully.
Added user `dbfconsole` successfully.
Added user `dbfukbox` successfully.
Attached Policies: [dbf-console]
To User: dbfconsole
Attached Policies: [dbf-ukbox]
To User: dbfukbox
=== RUN   TestDBForwardMinio
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 20:38:20 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=720
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 20:38:20 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=370
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 20:38:20 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=454
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 20:38:20 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=720
2026/10/24 20:38:20 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 20:38:20 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/24 20:38:20 console: could not notify member@example.com of a CLI token issue: keymail: lookup member@example.com: keymail: GET /api/federation/lookup?addr=member%40example.com&from=console%40127.0.0.1%3A37591: could not determine target mailbox
    agent.go:135: dbreads: dbr_fpzctxm3ahqt55amzmkcakngai: planned 1 files (720 bytes) and restored to 1 in 8ms
    agent.go:135: dbreads: dbr_fpzctxm3ahqt55amzmkcakngai: output 5 bytes in 1 confirmed chunks in 1ms
2026/10/24 20:38:21 console: db read dbr_fpzctxm3ahqt55amzmkcakngai: relayed 1 chunks (102 bytes) in 1.004s, ended done
    agent.go:135: dbreads: dbr_krzncnxlbp6rt4rlryvwocyt2y: planned 3 files (1544 bytes) and restored to 3 in 9ms
    agent.go:135: dbreads: dbr_krzncnxlbp6rt4rlryvwocyt2y: output 5 bytes in 1 confirmed chunks in 1ms
2026/10/24 20:38:22 console: db read dbr_krzncnxlbp6rt4rlryvwocyt2y: relayed 1 chunks (102 bytes) in 1.003s, ended done
    agent.go:135: dbreads: dbr_7sy35do7osmjpuicu7wjcrt36m: planned 3 files (1544 bytes) and held through 3 in 8ms
    agent.go:135: dbreads: dbr_7sy35do7osmjpuicu7wjcrt36m: output 1544 bytes in 1 confirmed chunks in 1ms
2026/10/24 20:38:23 console: db read dbr_7sy35do7osmjpuicu7wjcrt36m: relayed 1 chunks (1641 bytes) in 1.004s, ended done
=== RUN   TestDBForwardMinio/by_disconnect
=== NAME  TestDBForwardMinio
    agent.go:135: dbreads: dbr_hvxiir5fvdu672dsogdlmivav4: planned 3 files (1544 bytes) and restored to 3 in 9ms
2026/10/24 20:38:23 console: db read dbr_hvxiir5fvdu672dsogdlmivav4: relayed 0 chunks (0 bytes) in 52ms, ended client gone
    agent.go:135: dbreads: dbr_hvxiir5fvdu672dsogdlmivav4: output 5 bytes in 1 confirmed chunks in 103ms
=== RUN   TestDBForwardMinio/by_timeout
=== NAME  TestDBForwardMinio
    agent.go:135: dbreads: dbr_qbqbqjowfz4p24x47avm7jnxbm: planned 3 files (1544 bytes) and restored to 3 in 9ms
2026/10/24 20:38:44 console: db read dbr_qbqbqjowfz4p24x47avm7jnxbm: relayed 0 chunks (0 bytes) in 19.904s, ended no-result
    agent.go:135: dbreads: dbr_qbqbqjowfz4p24x47avm7jnxbm: output 5 bytes in 1 confirmed chunks in 19.88s
--- PASS: TestDBForwardMinio (23.35s)
    --- PASS: TestDBForwardMinio/by_disconnect (0.24s)
    --- PASS: TestDBForwardMinio/by_timeout (20.01s)
PASS
ok  	github.com/carlosframework/platform/internal/console	23.371s
minio contract: PASS
diff --git a/AGENTS.md b/AGENTS.md
index f3637c3d5..59be1f922 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -222,6 +222,50 @@ anything.
GOPROXY=off`; go commands may need the sandbox disabled ("read-only file
system" from the toolchain is the tell).
+## The census: an approval, not a decision (Paul, 2026-10-24, #369)
+
+`TestPlan14RootReadCensus` (`internal/provision`) fails when a branch
+changes a unit that reads root-owned state, or a package inside the
+census envelope. Every such change needs the census owner's approval on
+#280 before the merge. Most are approvals: the same reads in an edited
+body, or an envelope fingerprint that moved. Get them approved like this:
+
+1. `make census-card` prints what changed and writes nothing. Each change
+ has a class:
+ - `body`: the digest moved and the reads did not;
+ - `sites`: more or fewer reads of kinds the unit already makes;
+ - `envelope`: a package was re-pinned;
+ - `reads` (a kind of read the unit did not make before), `new`,
+ `removed` or `other` (any census discrepancy none of these explains,
+ shown with the census's own message): a decision.
+2. Write a CLAIMS file with one line per changed unit:
+ `<file>:<unit><TAB><what changed, and that no read was added or which one was>`.
+3. `make census-propose NAME=<branch> CLAIMS=<file>` writes the rest:
+ - the overlay (`testdata/plan14-root-reads-<NAME>.tsv`), each unit
+ re-frozen at its existing disposition with the claim appended;
+ - the `@legacy` fingerprints, the envelope re-pins and the new seal;
+ - its line in `testdata/plan14-root-reads-chain.tsv`;
+ - the approval card, `docs/superpowers/notes/census-proposal-<NAME>.md`.
+
+ It checks that the overlay leaves the census clean, and writes nothing
+ if any step fails. If the code changes again, rerun it with the same
+ NAME: it replaces its own overlay (only one it wrote itself), which must
+ still be the last line of the chain.
+4. Show the card to the census owner and ask for an approval. Ask in the
+ session when they are there (one question: approve or hold); otherwise
+ post the card on #280. Record the approval on #280: who, when, and the
+ tip. Rows carry `rev3=census-ruling-<NAME>` from the start, so there is
+ nothing to flip afterwards and no second CI run.
+
+The tool refuses decisions (`reads`, `new`, `removed`, `other`): a new
+kind of read can make a unit's disposition wrong, so it is the owner's
+call, not an approval. Write those rows by hand, following the overlays in the
+chain, and ask on #280 with the card attached; the census owner decides
+those. Nothing about the census's strength changes: every changed read is
+still named, digested and seen by a person before merge. The record-write
+census (9B8, the `*_census_test.go` inventories) still takes hand-written
+entries.
+
## Rules this repo has paid for
- **A rule stated as an absence can only be checked by looking for the absent
diff --git a/Makefile b/Makefile
index fd9a7f63f..efae74f45 100644
--- a/Makefile
+++ b/Makefile
@@ -3,7 +3,7 @@ LDFLAGS := -s -w -X github.com/carlosframework/platform/internal/version.Build=$
GOOS ?=
GOARCH ?=
-.PHONY: test build fmt
+.PHONY: test build fmt census-card census-propose
# The build gate: vet, format, tests. `.amadan/ci.d/10-gate` runs this same
# target, so what CI checks and what you check are one definition.
@@ -23,3 +23,24 @@ build:
fmt:
gofmt -w .
+
+# The root-read census as an approval (#369, AGENTS.md "The census"):
+# census-card prints what changed and writes nothing; census-propose writes
+# the overlay, its chain manifest line and the approval card. CLAIMS has
+# one line per changed unit, "<file>:<unit><TAB><what changed>". Kept here
+# rather than in a script: every .sh in the tree is itself a census unit.
+CENSUS_RUN = out=$$(mktemp "$${TMPDIR:-/tmp}/census.XXXXXX") || exit 1; \
+ trap 'rm -f "$$out"' EXIT; \
+ GOFLAGS="$${GOFLAGS:--buildvcs=false}" go test ./internal/provision/ -run '^TestPlan14RootReadCensusPropose$$' -count=1 -v >"$$out" 2>&1; rc=$$?; \
+ if [ $$rc -eq 0 ]; then sed -n '/^--- census card ---$$/,/^--- end census card ---$$/p' "$$out" | sed '1d;$$d'; \
+ else grep -v '^20[0-9][0-9]/' "$$out" | tail -20 >&2; fi; exit $$rc
+# CLAIMS is kept as one value, spaces and all, and made absolute for go test.
+CENSUS_CLAIMS = c='$(CLAIMS)'; if [ -n "$$c" ]; then case "$$c" in /*) ;; *) c="$$PWD/$$c" ;; esac; \
+ CARLOS_CENSUS_CLAIMS="$$c"; export CARLOS_CENSUS_CLAIMS; fi
+
+census-card:
+ @CARLOS_CENSUS_CARD=1; export CARLOS_CENSUS_CARD; $(CENSUS_CLAIMS); $(CENSUS_RUN)
+
+census-propose:
+ @if [ -z '$(NAME)' ] || [ -z '$(CLAIMS)' ]; then echo "usage: make census-propose NAME=<branch> CLAIMS=<file>" >&2; exit 2; fi; \
+ CARLOS_CENSUS_PROPOSE='$(NAME)'; export CARLOS_CENSUS_PROPOSE; $(CENSUS_CLAIMS); $(CENSUS_RUN)
diff --git a/README.md b/README.md
index 7a4b80895..c6ec2a3ac 100644
--- a/README.md
+++ b/README.md
@@ -644,6 +644,13 @@ publish an inflated instance count.
`carlos system geoip publish` instead (docs/site/cli.md, "Platform GeoIP").
Uploads are staged in `geoip-staging/` next to `CONSOLE_DB`, which the
console clears at startup.
+- `CARLOS_SIDECAR_ACCOUNTS`: comma-separated account sqids (not names) that
+ may still enable and create native sidecars. Set, every other account gets
+ 403 `sidecars-paused` from `carlos sidecars enable` and `carlos sidecars
+ create`. List, get, wake and delete still work, and existing sidecars keep
+ running. Native sidecars run on the flat tier, outside the nested tenants'
+ isolation, until nested sidecars land (discussion #370). **Unset or empty =
+ no gate.** An entry that is not a sqid stops the console at boot.
**Canonical-host caveat**, stated honestly: an app that pins its own host
(e.g. keymail's `--host keymail.dev`) will redirect or 404 on its platform
diff --git a/cmd/console/main.go b/cmd/console/main.go
index c95cfa05f..067f138c9 100644
--- a/cmd/console/main.go
+++ b/cmd/console/main.go
@@ -51,6 +51,14 @@
// CARLOS_TEMPORARY_APPS=1 turns on temporary apps (carlos deploy
// --temporary); off by default, see the temporary static sites rollout.
//
+// CARLOS_SIDECAR_ACCOUNTS (comma-separated account sqids) pauses native
+// sidecars for every account it does not name: enabling sidecars and
+// creating one answer 403 sidecars-paused, while list, get, wake and delete
+// keep working and existing sidecars keep running. Native sidecars run on
+// the flat tier, outside the nested tenants' isolation, until discussion
+// #370 lands. Unset or empty is no gate. An entry that is not a sqid, or a
+// value that names no sqid at all, stops the console at startup.
+//
// Tenant email (task 17) reads CARLOS_MAIL_OPERATORS (comma-separated
// addresses; narrows the fleet-operator bit further for the pause/cap
// door only — see mailOperators' doc comment), CARLOS_MAIL_REGION,
@@ -107,6 +115,7 @@ import (
"github.com/carlosframework/platform/internal/appconfig"
"github.com/carlosframework/platform/internal/console"
"github.com/carlosframework/platform/internal/dnsrec"
+ "github.com/carlosframework/platform/internal/domains"
"github.com/carlosframework/platform/internal/geo"
"github.com/carlosframework/platform/internal/geoip"
"github.com/carlosframework/platform/internal/lease"
@@ -291,6 +300,12 @@ func serve() {
} else {
log.Print("console: CARLOS_RELEASE_KEY unset — promotions disabled, running read-only")
}
+ // Before anything is opened: a typo in the allowlist must stop the
+ // console rather than quietly pause, or unpause, the wrong accounts.
+ sidecarAccts, err := sidecarAccounts(os.Getenv)
+ if err != nil {
+ log.Fatalf("console: %v", err)
+ }
hostClaim := hostClaimEnabled(os.Getenv)
st := openServeDB(hostClaim)
storeProvisioner, storeDeployment := storeBackend()
@@ -457,6 +472,9 @@ func serve() {
// owners (discussion #176). Unset is the owner-only console.
SystemOperators: systemOperators(),
PlatformDomain: platformDomain(),
+ // The sidecar pause (internal/console/sidecarsgate.go). Unset is no
+ // gate; set, only these accounts may enable or create a sidecar.
+ SidecarAccounts: sidecarAccts,
}
// The Phase 1 moves switch (CARLOS_MOVES_PHASE1, off by default, index
// D13): applied after the literal so a test can prove it reaches the
@@ -875,6 +893,42 @@ func systemOperators() []string {
return out
}
+// sidecarAccounts parses CARLOS_SIDECAR_ACCOUNTS (comma-separated account
+// sqids): the accounts that may still enable and create native sidecars
+// while every other account is paused (internal/console/sidecarsgate.go,
+// discussion #370). Each entry is trimmed and lower-cased, and must be a
+// sqid (domains.ValidSqid), never an account name: "ops" is not one, and a
+// name that matched nothing would pause the very account it meant to keep.
+// An invalid entry is an error for serve to stop on.
+//
+// Unset or the empty string is nil, and nil is no gate: every account may
+// enable and create sidecars, exactly as before the list existed. This is
+// the opposite direction to the operator lists above on purpose, so a
+// deployment that never sets it is unaffected. Any other value that names
+// no sqid at all ("," or " , ") is an error too: it reads as a list, and
+// parsing it to nil would quietly turn the gate off.
+func sidecarAccounts(getenv func(string) string) ([]string, error) {
+ raw := getenv("CARLOS_SIDECAR_ACCOUNTS")
+ if raw == "" {
+ return nil, nil
+ }
+ var out []string
+ for _, a := range strings.Split(raw, ",") {
+ a = strings.ToLower(strings.TrimSpace(a))
+ if a == "" {
+ continue
+ }
+ if !domains.ValidSqid(a) {
+ return nil, fmt.Errorf("CARLOS_SIDECAR_ACCOUNTS: %q is not an account sqid (the account's id, e.g. bab, never its name)", a)
+ }
+ out = append(out, a)
+ }
+ if len(out) == 0 {
+ return nil, fmt.Errorf("CARLOS_SIDECAR_ACCOUNTS: %q names no account sqid; unset it for no gate, or list at least one sqid", raw)
+ }
+ return out, nil
+}
+
// platformDomain reads CARLOS_PLATFORM_DOMAIN — the domain the relay's own
// hostnames hang off (relay.<region>.<domain>), which an enabled app is
// handed as its ICE server URLs.
diff --git a/cmd/console/sidecaraccounts_serve_test.go b/cmd/console/sidecaraccounts_serve_test.go
new file mode 100644
index 000000000..066138fba
--- /dev/null
+++ b/cmd/console/sidecaraccounts_serve_test.go
@@ -0,0 +1,55 @@
+package main_test
+
+import (
+ "context"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+)
+
+// TestServeStopsOnABadSidecarAccount: serve reads CARLOS_SIDECAR_ACCOUNTS
+// and stops before it opens anything on an entry that is not a sqid, or on a
+// value set to a list that names no sqid at all ("," would otherwise parse
+// to nil and quietly turn the gate off). The timeout is the failure mode it
+// guards: a serve that ignored the variable would carry on and listen.
+func TestServeStopsOnABadSidecarAccount(t *testing.T) {
+ bin := buildConsole(t)
+ for _, c := range []struct{ val, want string }{
+ {"bab,ops", `"ops"`},
+ {",", "names no account sqid"},
+ {" , ", "names no account sqid"},
+ } {
+ tmp := t.TempDir()
+ ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ cmd := exec.CommandContext(ctx, bin, "serve")
+ cmd.Dir = tmp
+ cmd.Env = append(os.Environ(),
+ "CONSOLE_ORIGIN=http://127.0.0.1:1",
+ "CONSOLE_ADDR=127.0.0.1:0",
+ "CONSOLE_DB="+filepath.Join(tmp, "console.db"),
+ // 32 zero bytes: a well-formed Ed25519 public key, so serve gets
+ // past the key check to the variable under test.
+ "CARLOS_RELEASE_PUBKEY=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
+ "CARLOS_RELEASE_KEY=",
+ "CARLOS_SIDECAR_ACCOUNTS="+c.val,
+ )
+ out, err := cmd.CombinedOutput()
+ timedOut := ctx.Err() != nil
+ cancel()
+ if timedOut {
+ t.Fatalf("CARLOS_SIDECAR_ACCOUNTS=%q: serve did not stop: %s", c.val, out)
+ }
+ if err == nil {
+ t.Fatalf("CARLOS_SIDECAR_ACCOUNTS=%q: serve exited 0: %s", c.val, out)
+ }
+ if !strings.Contains(string(out), "CARLOS_SIDECAR_ACCOUNTS") || !strings.Contains(string(out), c.want) {
+ t.Fatalf("CARLOS_SIDECAR_ACCOUNTS=%q: the refusal does not name the variable and %s: %s", c.val, c.want, out)
+ }
+ if _, err := os.Stat(filepath.Join(tmp, "console.db")); err == nil {
+ t.Fatalf("CARLOS_SIDECAR_ACCOUNTS=%q: serve opened its database before refusing", c.val)
+ }
+ }
+}
diff --git a/cmd/console/sidecaraccounts_test.go b/cmd/console/sidecaraccounts_test.go
new file mode 100644
index 000000000..cba72c2bd
--- /dev/null
+++ b/cmd/console/sidecaraccounts_test.go
@@ -0,0 +1,44 @@
+package main
+
+import (
+ "slices"
+ "testing"
+)
+
+// CARLOS_SIDECAR_ACCOUNTS reaches Config.SidecarAccounts through
+// sidecarAccounts: trimmed, lower-cased, each a sqid. Unset and the empty
+// string are nil, which is no gate; any entry that is not a sqid, and any
+// other value that names no sqid, is an error serve stops on (TestServeStopsOnABadSidecarAccount runs that through the binary).
+func TestSidecarAccountsParsesTheList(t *testing.T) {
+ for _, tc := range []struct {
+ val string
+ want []string
+ wantErr bool
+ }{
+ {"", nil, false},
+ {" ", nil, true}, // set, but names no sqid: refused, never "no gate"
+ {",", nil, true}, // the same, and the review's case
+ {" , ", nil, true}, // ditto
+ {"bab", []string{"bab"}, false},
+ {" BAB , cdef ,", []string{"bab", "cdef"}, false},
+ {"bab,ops", nil, true}, // a name, not a sqid: o is not in the alphabet
+ {"bab,ab", nil, true}, // too short
+ {"bab,b-b", nil, true}, // not a letter
+ {"bab;cdef", nil, true}, // wrong separator reads as one bad entry
+ {"x1234567890abcdefg", nil, true},
+ } {
+ got, err := sidecarAccounts(func(k string) string {
+ if k != "CARLOS_SIDECAR_ACCOUNTS" {
+ t.Fatalf("read %s", k)
+ }
+ return tc.val
+ })
+ if (err != nil) != tc.wantErr {
+ t.Errorf("CARLOS_SIDECAR_ACCOUNTS=%q: err %v, want error %v", tc.val, err, tc.wantErr)
+ continue
+ }
+ if !slices.Equal(got, tc.want) {
+ t.Errorf("CARLOS_SIDECAR_ACCOUNTS=%q: %v, want %v", tc.val, got, tc.want)
+ }
+ }
+}
diff --git a/docs/site/cli.md b/docs/site/cli.md
index 0a8e384f2..3d0dd287a 100644
--- a/docs/site/cli.md
+++ b/docs/site/cli.md
@@ -1016,6 +1016,12 @@ it back from, so `carlos sidecars ls` shows the build its box last started
for each sidecar, beside the version its channel points at. One that has not
started on any box reads as unknown rather than as current.
+A console operator can pause sidecars for every account not named in
+`CARLOS_SIDECAR_ACCOUNTS`, a comma-separated list of account sqids. A paused
+account gets a 403 with the code `sidecars-paused` from `enable` and
+`create`. Its existing sidecars keep running, and `ls`, `wake` and `rm` still
+work. Unset, nothing is paused.
+
### carlos steering
Decides whether one instance's DNS answer varies by where the request comes
diff --git a/docs/superpowers/notes/census-proposal-console-brand-name.md b/docs/superpowers/notes/census-proposal-console-brand-name.md
new file mode 100644
index 000000000..e2e8ccd7d
--- /dev/null
+++ b/docs/superpowers/notes/census-proposal-console-brand-name.md
@@ -0,0 +1,13 @@
+# Census: console brand name
+
+Status: APPROVED by Paul (census owner), 2026-10-24, recorded on #280 with
+main at 65ca4aca. Rows keep `rev3=PROPOSED-console-brand-name` (no flip under
+#369).
+
+Overlay `internal/provision/testdata/plan14-root-reads-console-brand-name.tsv`,
+landed with 5f341f3. The console's chrome and new-account welcome name the
+deployment from `CARLOS_CONSOLE_NAME`. Three embedded templates change text
+only (`apps.html`, `home.html`, `layout.html`), and the `internal/console` and
+`internal/console/templates` envelopes are re-pinned. No read, file or record
+write is added. The units run in the console (`cmd/console serve` under
+`console.service`, user carlos-console).
diff --git a/docs/superpowers/notes/census-proposal-email-signin.md b/docs/superpowers/notes/census-proposal-email-signin.md
index f1f186c70..c32d7eb3a 100644
--- a/docs/superpowers/notes/census-proposal-email-signin.md
+++ b/docs/superpowers/notes/census-proposal-email-signin.md
@@ -1,7 +1,9 @@
# Census: console email sign-in
-Status: PROPOSED. Every row carries `rev3=PROPOSED-email-signin`. The ruling
-is asked of the census owner on #280.
+Status: APPROVED by Paul (census owner), 2026-10-24, recorded on #280 with
+main at 65ca4aca. The rows keep `rev3=PROPOSED-email-signin`: under the #369
+process nothing is flipped, and later overlays re-froze units at these
+dispositions, so a flip would move their fingerprints.
Branch `email-signin`, cut from main at 4d6f0e6c. Design:
`docs/superpowers/specs/2026-10-21-console-email-signin-design.md`.
diff --git a/docs/superpowers/notes/census-proposal-sec-deps-dos-bumps.md b/docs/superpowers/notes/census-proposal-sec-deps-dos-bumps.md
new file mode 100644
index 000000000..97d2e1f04
--- /dev/null
+++ b/docs/superpowers/notes/census-proposal-sec-deps-dos-bumps.md
@@ -0,0 +1,37 @@
+# Census approval: sec-deps-dos-bumps
+
+Ruling `census-ruling-sec-deps-dos-bumps`. Every change below keeps the disposition it already has, so it needs an approval, not a decision.
+
+Envelopes re-pinned: `.`.
+
+Seal: `945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a` → `945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a` (unchanged; loaded after sec-host-imdsv2-module, re-chained 2026-10-24 when that merged as ef156d3).
+
+Overlay: `internal/provision/testdata/plan14-root-reads-sec-deps-dos-bumps.tsv`, appended to `internal/provision/testdata/plan14-root-reads-chain.tsv`.
+
+## Branch context
+
+Branch `sec-deps-dos-bumps` bumps three modules that govulncheck reports as
+reachable on main:
+
+| module | main | branch | findings |
+|---|---|---|---|
+| `golang.org/x/net` | v0.56.0 | v0.60.0 | GO-2026-6603, -6610, -6611, -6612, -6617 (HTTP/2 server and transport DoS) |
+| `golang.org/x/text` | v0.38.0 | v0.42.0 | GO-2026-5970 (idna infinite loop, reached from the edge's TLS SNI handling) |
+| `github.com/pion/stun/v3` | v3.0.1 | v3.1.5 | GO-2026-6163 (panic on a malformed XOR-MAPPED-ADDRESS, reached from the relay's TURN server) |
+
+x/text lands at v0.42.0, not v0.39.0, because x/net v0.60.0 requires it. The
+same requirement moves x/crypto (v0.57.0), x/sys (v0.48.0), x/sync (v0.23.0),
+x/mod (v0.41.0) and x/tools (v0.49.0), and raises go.mod's `go` directive from
+`1.26` to `1.26.0`. stun v3.1.5 moves pion/dtls/v3 to v3.1.4 and
+pion/transport/v4 to v4.0.2, and drops pion/transport/v3. pion/turn/v4 stays at
+v4.1.4.
+
+The change is go.mod and go.sum only. No Go source file changes, so no read is
+added, removed or changed, no census unit's digest moves, and the legacy seal
+does not move. The only census effect is the envelope re-pin of `.` above.
+The record-write census (9B8) and claim routes are unchanged.
+
+go.mod meets `go-1.26.10-emudhra`, which raises the same `go` directive to
+`1.26.10` and pins `.` in its own overlay. Whichever lands second takes
+`1.26.10` and re-pins `.` once more (rerun `make census-propose` with the same
+NAME).
diff --git a/docs/superpowers/notes/census-proposal-sec-host-imdsv2-module.md b/docs/superpowers/notes/census-proposal-sec-host-imdsv2-module.md
new file mode 100644
index 000000000..37512514f
--- /dev/null
+++ b/docs/superpowers/notes/census-proposal-sec-host-imdsv2-module.md
@@ -0,0 +1,28 @@
+# Census approval: sec-host-imdsv2-module
+
+Ruling `census-ruling-sec-host-imdsv2-module`. Every change below keeps the disposition it already has, so it needs an approval, not a decision.
+
+| unit | change | read sites | what changed |
+|---|---|---|---|
+| `infra/modules/region-host/main.tf` `external` | body | unchanged (1) | aws_instance.host gains metadata_options (IMDSv2 required, hop limit 1, endpoint enabled), matching scanner-host; tofu configuration only, run on an operator's machine; no read, file or record write is added; applies in place on a running instance with no restart |
+
+Envelopes re-pinned: `infra/modules/region-host`.
+
+Seal: `09e20ea4bbc534710334e632e903878656dabfb5817d4d96c6e7c56508172148` → `945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a`.
+
+Overlay: `internal/provision/testdata/plan14-root-reads-sec-host-imdsv2-module.tsv`, appended to `internal/provision/testdata/plan14-root-reads-chain.tsv`.
+
+## Branch context
+
+The branch `sec-host-imdsv2-module` adds a `metadata_options` block to
+`aws_instance.host` in `infra/modules/region-host/main.tf` (IMDSv2 required,
+hop limit 1, endpoint enabled), matching scanner-host, and a plan-time tofu
+test under `infra/modules/region-host/tests/`. The record-write census does
+not change: no record write is added or moved.
+
+The new `tests/instance_metadata.tftest.hcl` is outside the census: the scan's
+extension list excludes `.tftest.hcl`, and envelopes group by immediate
+directory, so the module hash covers only the `.tf` files.
+
+Executing identity: tofu on an operator's machine with the operator's AWS
+credentials; no box unit runs the file and no binary embeds it.
diff --git a/docs/superpowers/notes/census-proposal-sec-sidecar-create-gate.md b/docs/superpowers/notes/census-proposal-sec-sidecar-create-gate.md
new file mode 100644
index 000000000..7fe04527e
--- /dev/null
+++ b/docs/superpowers/notes/census-proposal-sec-sidecar-create-gate.md
@@ -0,0 +1,13 @@
+# Census approval: sec-sidecar-create-gate
+
+Ruling `census-ruling-sec-sidecar-create-gate`. Every change below keeps the disposition it already has, so it needs an approval, not a decision.
+
+| unit | change | read sites | what changed |
+|---|---|---|---|
+| `cmd/console/main.go` `serve` | body | unchanged (3) | serve now parses CARLOS_SIDECAR_ACCOUNTS (sidecarAccounts, a pure env parse with log.Fatalf on a bad or empty-list value) before opening anything and passes it into console.Config.SidecarAccounts. No read was added: the same 3 root read sites. |
+
+Envelopes re-pinned: `cmd/console`, `internal/console`.
+
+Seal: `945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a` → `0ca3baafa26f59c0b7d8c6d6a39237d074afb71f885b7fbd50debebab5bf89f2`.
+
+Overlay: `internal/provision/testdata/plan14-root-reads-sec-sidecar-create-gate.tsv`, appended to `internal/provision/testdata/plan14-root-reads-chain.tsv`.
diff --git a/go.mod b/go.mod
index 5a0e3cad5..c320d08df 100644
--- a/go.mod
+++ b/go.mod
@@ -1,6 +1,6 @@
module github.com/carlosframework/platform
-go 1.26
+go 1.26.0
require (
github.com/aws/aws-sdk-go-v2 v1.44.0
@@ -21,12 +21,12 @@ require (
github.com/aws/smithy-go v1.28.1
github.com/benbjohnson/litestream v0.5.14
github.com/pion/logging v0.2.4
- github.com/pion/stun/v3 v3.0.1
+ github.com/pion/stun/v3 v3.1.5
github.com/pion/turn/v4 v4.1.4
github.com/superfly/ltx v0.5.1
- golang.org/x/crypto v0.53.0
- golang.org/x/net v0.56.0
- golang.org/x/sys v0.46.0
+ golang.org/x/crypto v0.57.0
+ golang.org/x/net v0.60.0
+ golang.org/x/sys v0.48.0
modernc.org/sqlite v1.55.0
)
@@ -60,10 +60,9 @@ require (
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
- github.com/pion/dtls/v3 v3.0.7 // indirect
+ github.com/pion/dtls/v3 v3.1.4 // indirect
github.com/pion/randutil v0.1.0 // indirect
- github.com/pion/transport/v3 v3.0.8 // indirect
- github.com/pion/transport/v4 v4.0.1 // indirect
+ github.com/pion/transport/v4 v4.0.2 // indirect
github.com/prometheus/client_golang v1.17.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
@@ -73,8 +72,9 @@ require (
github.com/tetratelabs/wazero v1.2.1 // indirect
github.com/wasilibs/go-re2 v1.3.0 // indirect
github.com/wlynxg/anet v0.0.5 // indirect
- golang.org/x/sync v0.21.0 // indirect
- golang.org/x/text v0.38.0 // indirect
+ golang.org/x/mod v0.41.0 // indirect
+ golang.org/x/sync v0.23.0 // indirect
+ golang.org/x/text v0.42.0 // indirect
google.golang.org/protobuf v1.36.10 // indirect
modernc.org/libc v1.74.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
@@ -86,8 +86,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/sns v1.43.0
github.com/klauspost/compress v1.18.0
github.com/oschwald/maxminddb-golang v1.13.1
- golang.org/x/mod v0.37.0
- golang.org/x/tools v0.47.0
+ golang.org/x/tools v0.49.0
)
replace amadan.net/carlos/platform.git/launchctx => ./launchctx
diff --git a/go.sum b/go.sum
index 4558d7955..a7dd6e987 100644
--- a/go.sum
+++ b/go.sum
@@ -154,18 +154,16 @@ github.com/oschwald/maxminddb-golang v1.13.1 h1:G3wwjdN9JmIK2o/ermkHM+98oX5fS+k5
github.com/oschwald/maxminddb-golang v1.13.1/go.mod h1:K4pgV9N/GcK694KSTmVSDTODk4IsCNThNdTmnaBZ/F8=
github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=
github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
-github.com/pion/dtls/v3 v3.0.7 h1:bItXtTYYhZwkPFk4t1n3Kkf5TDrfj6+4wG+CZR8uI9Q=
-github.com/pion/dtls/v3 v3.0.7/go.mod h1:uDlH5VPrgOQIw59irKYkMudSFprY9IEFCqz/eTz16f8=
+github.com/pion/dtls/v3 v3.1.4 h1:QhvtMflMfu9Kf0RcDC5BJBle4caPskByrKQR6uuYqpY=
+github.com/pion/dtls/v3 v3.1.4/go.mod h1:cr/qotLISUw/9C1m83ZPNZtj9WnXkYLpfCptPqbkInc=
github.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=
github.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=
github.com/pion/randutil v0.1.0 h1:CFG1UdESneORglEsnimhUjf33Rwjubwj6xfiOXBa3mA=
github.com/pion/randutil v0.1.0/go.mod h1:XcJrSMMbbMRhASFVOlj/5hQial/Y8oH/HVo7TBZq+j8=
-github.com/pion/stun/v3 v3.0.1 h1:jx1uUq6BdPihF0yF33Jj2mh+C9p0atY94IkdnW174kA=
-github.com/pion/stun/v3 v3.0.1/go.mod h1:RHnvlKFg+qHgoKIqtQWMOJF52wsImCAf/Jh5GjX+4Tw=
-github.com/pion/transport/v3 v3.0.8 h1:oI3myyYnTKUSTthu/NZZ8eu2I5sHbxbUNNFW62olaYc=
-github.com/pion/transport/v3 v3.0.8/go.mod h1:+c2eewC5WJQHiAA46fkMMzoYZSuGzA/7E2FPrOYHctQ=
-github.com/pion/transport/v4 v4.0.1 h1:sdROELU6BZ63Ab7FrOLn13M6YdJLY20wldXW2Cu2k8o=
-github.com/pion/transport/v4 v4.0.1/go.mod h1:nEuEA4AD5lPdcIegQDpVLgNoDGreqM/YqmEx3ovP4jM=
+github.com/pion/stun/v3 v3.1.5 h1:Y1FHlhaI6+4UoC5i/zQf4F7JvdZtB24/05oyy/GF1x8=
+github.com/pion/stun/v3 v3.1.5/go.mod h1:zRUghXSQU32Lx5orJsz3uYMkIihweXb3mu5gIns02fs=
+github.com/pion/transport/v4 v4.0.2 h1:ifYlPqNwsy6aKQ9y8yzxXlHae5431ZrH2avkD/Rn6Tk=
+github.com/pion/transport/v4 v4.0.2/go.mod h1:06hFI+jCFcok2X2MekVufNZ/uzNZXivGBPfviSVcjgM=
github.com/pion/turn/v4 v4.1.4 h1:EU11yMXKIsK43FhcUnjLlrhE4nboHZq+TXBIi3QpcxQ=
github.com/pion/turn/v4 v4.1.4/go.mod h1:ES1DXVFKnOhuDkqn9hn5VJlSWmZPaRJLyBXoOeO/BmQ=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
@@ -214,35 +212,25 @@ go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF
go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
-golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
-golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
-golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
-golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
-golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
-golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
-golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
-golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
-golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
-golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
-golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
-golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
+golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
+golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
+golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
+golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
+golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU=
+golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw=
golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
-golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
-golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
+golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
+golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
-golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
-golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
-golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
-golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
-golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
-golang.org/x/time v0.10.0 h1:3usCWA8tQn0L8+hFJQNgzpWbd89begxN66o1Ojdn5L4=
-golang.org/x/time v0.10.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
-golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
-golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
-golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
-golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
+golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
+golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
+golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
+golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
+golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
+golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
+golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
+golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.155.0 h1:vBmGhCYs0djJttDNynWo44zosHlPvHmA0XiN2zP2DtA=
google.golang.org/api v0.155.0/go.mod h1:GI5qK5f40kCpHfPn6+YzGAByIKWv8ujFnmoWm7Igduk=
diff --git a/infra/modules/region-host/main.tf b/infra/modules/region-host/main.tf
index b35ecc196..11fca1987 100644
--- a/infra/modules/region-host/main.tf
+++ b/infra/modules/region-host/main.tf
@@ -490,6 +490,19 @@ resource "aws_instance" "host" {
iam_instance_profile = aws_iam_instance_profile.host.name
user_data = var.user_data
user_data_replace_on_change = false
+ # IMDSv2 only, one hop. With the block absent EC2 leaves IMDSv1 on, and a
+ # plain GET from any request-forgery bug in a tenant app reads this
+ # role's credentials; v2 needs a PUT with a header first, which those
+ # bugs rarely give an attacker. Hop limit 1 keeps the token from leaving
+ # the box. Changing these on a running instance is in place, no restart.
+ # The flagship boxes were switched by hand on 2026-10-23; this keeps a
+ # replaced box from coming back on v1. The scanner host has done the same
+ # since it was written.
+ metadata_options {
+ http_endpoint = "enabled"
+ http_tokens = "required"
+ http_put_response_hop_limit = 1
+ }
root_block_device {
volume_size = var.root_volume_size
volume_type = "gp3"
diff --git a/infra/modules/region-host/tests/instance_metadata.tftest.hcl b/infra/modules/region-host/tests/instance_metadata.tftest.hcl
new file mode 100644
index 000000000..a2c60aa2e
--- /dev/null
+++ b/infra/modules/region-host/tests/instance_metadata.tftest.hcl
@@ -0,0 +1,49 @@
+# Plan-time tests for the host instance's metadata service settings.
+# mock_provider means no AWS account and no credentials.
+mock_provider "aws" {}
+
+override_data {
+ target = data.aws_vpcs.default
+ values = { ids = ["vpc-0aaaaaaaaaaaaaaaa"] }
+}
+
+override_data {
+ target = data.aws_caller_identity.current
+ values = { account_id = "123456789012" }
+}
+
+# The termination queue's ARN feeds an EventBridge target, which the provider
+# validates as an ARN; the mock's random string is not one.
+override_resource {
+ target = aws_sqs_queue.terminations
+ values = { arn = "arn:aws:sqs:eu-west-1:123456789012:carlos-test-host-terminations" }
+}
+
+variables {
+ name = "carlos-test-ie"
+ sg_description = "carlos test host"
+ iam_name = "carlos-test-host"
+ instance_type = "t4g.small"
+ root_volume_size = 20
+ user_data = "#!/bin/sh\n"
+ deployment_bucket_arn = "arn:aws:s3:::carlos-test-platform"
+ data_bucket_arn = "arn:aws:s3:::carlos-test-ie"
+ ssm_parameter_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/carlos/*"
+}
+
+
+run "imdsv2_only" {
+ command = plan
+ assert {
+ condition = aws_instance.host.metadata_options[0].http_tokens == "required"
+ error_message = "the host requires IMDSv2 tokens"
+ }
+ assert {
+ condition = aws_instance.host.metadata_options[0].http_put_response_hop_limit == 1
+ error_message = "the IMDS token must not leave the box (hop limit 1)"
+ }
+ assert {
+ condition = aws_instance.host.metadata_options[0].http_endpoint == "enabled"
+ error_message = "the endpoint stays on: the box role is how the host reaches AWS"
+ }
+}
diff --git a/internal/console/apisidecars.go b/internal/console/apisidecars.go
index 698b7daa4..17c8aa12d 100644
--- a/internal/console/apisidecars.go
+++ b/internal/console/apisidecars.go
@@ -198,7 +198,8 @@ func (s *Server) handleAPISidecarGet(w http.ResponseWriter, r *http.Request) {
// handleAPISidecarCreate is the point of the feature: one create-only write
// to the bucket, in the family's refusal order — gate, budget, readiness
-// (instanceGate), manifest, name, region, channel, shape, write.
+// (instanceGate), manifest, name, region, channel, shape, pause
+// (sidecarsgate.go), write.
//
// Every field policy governs is stamped from the MANIFEST, never from the
// request. The body has no pool field to lie in, no host field (the host is
@@ -311,6 +312,12 @@ func (s *Server) handleAPISidecarCreate(w http.ResponseWriter, r *http.Request)
jsonError(w, http.StatusBadRequest, err.Error())
return
}
+ // The sidecar pause (sidecarsgate.go): after every check above, so a
+ // stranger still meets the 404 and a bad request its 400, and before
+ // the cap's listing, the leftover rail and either write arm.
+ if s.refuseSidecarsPaused(w, acct, app, "create") {
+ return
+ }
// Plans 3a: the plan caps workers per app, under the account lock.
defer s.planLock(acct)()
if err := s.checkSidecars(r.Context(), acct, app); err != nil {
@@ -586,6 +593,11 @@ func (s *Server) handleAPISidecarEnable(w http.ResponseWriter, r *http.Request)
return
}
}
+ // The sidecar pause (sidecarsgate.go): after authentication, membership
+ // and validation, before the manifest read and its write.
+ if s.refuseSidecarsPaused(w, acct, app, "enable") {
+ return
+ }
ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
defer cancel()
// Read WITH the etag and write guarded by it below: a read-modify-write
diff --git a/internal/console/server.go b/internal/console/server.go
index e7c940087..0c186db2f 100644
--- a/internal/console/server.go
+++ b/internal/console/server.go
@@ -855,6 +855,13 @@ type Config struct {
// wherever the two differ by anything but that label; the relay's DNS
// names are operator-managed and this console cannot discover them.
PlatformDomain string
+ // SidecarAccounts is the operator allowlist of account sqids that may
+ // still enable and create native sidecars (CARLOS_SIDECAR_ACCOUNTS,
+ // normalized by cmd/console: trimmed, lower-cased, each a valid sqid).
+ // Empty is no gate at all, every deployment's shape before the list
+ // existed. Set, every other account meets sidecarsPaused on the two
+ // doors that bring a sidecar into being (sidecarsgate.go).
+ SidecarAccounts []string
}
// Server is the console's http.Handler.
diff --git a/internal/console/sidecarsgate.go b/internal/console/sidecarsgate.go
new file mode 100644
index 000000000..002d4c8e5
--- /dev/null
+++ b/internal/console/sidecarsgate.go
@@ -0,0 +1,64 @@
+// The sidecar pause: an operator allowlist in front of the two doors that
+// bring a native sidecar into being.
+//
+// Why it exists: a native sidecar runs on the flat tier
+// (activator.ExecBackend.StartWorkerStaged) as the box's carlos uid with no
+// mount namespace, so on its box it can read and write every nested
+// tenant's /data/instances/<iid>. The fix is nested sidecars (amadan
+// discussion #370). Until that lands, an operator lists the accounts that
+// may still enable or create one (CARLOS_SIDECAR_ACCOUNTS) and every other
+// account is refused.
+//
+// The doors, and why there are only two. A sidecar runs only from a record
+// at apps/<sqid>/<app>/sidecars/<name>.json, which the box's sidecar
+// reconcile pass (or, under host claim, the root claim path) turns into a
+// Kind "sidecar" route, and it takes its argv and ceilings from
+// apps/<sqid>/<app>/sidecars.json. The console writes the record in exactly
+// one place (handleAPISidecarCreate, both the CreateSidecarRecord and the
+// host-claim beginClaimCreate arms) and the manifest in exactly one place
+// (handleAPISidecarEnable). Both call refuseSidecarsPaused. Ship and promote
+// publish releases and channel pointers, never a sidecar manifest; restore
+// clears an app's trash marker and brings back only sidecars that already
+// existed; carlos run needs a nested instance. Store mode (`carlos sidecars
+// enable` against the bucket) is the operator's own bucket credential and
+// is not a console door.
+//
+// What it leaves alone, on purpose: list, get, wake and delete. Existing
+// sidecars keep running, and an account taken off the list must still be
+// able to retire its own.
+package console
+
+import (
+ "log"
+ "net/http"
+ "slices"
+)
+
+// sidecarsPausedMsg is the one sentence a refused account reads.
+// copy: approved by Paul 2026-10-24 (copy review).
+const sidecarsPausedMsg = "sidecars are paused for this account. existing sidecars keep running, and you can still delete them"
+
+// sidecarsPausedCode is the refusal's machine-readable code, beside the
+// sentence the way plan caps carry "cap".
+const sidecarsPausedCode = "sidecars-paused"
+
+// sidecarsPaused reports whether acct is refused new sidecars: the
+// allowlist is set and does not name the account's sqid.
+func (s *Server) sidecarsPaused(acct *Account) bool {
+ return len(s.cfg.SidecarAccounts) > 0 && !slices.Contains(s.cfg.SidecarAccounts, acct.Sqid)
+}
+
+// refuseSidecarsPaused answers 403 sidecars-paused and reports true when
+// acct may not enable or create a sidecar. Call it AFTER the door's
+// authentication, membership and validation checks, so a non-member still
+// gets the byte-identical 404 and the gate never confirms that an account
+// exists, and BEFORE any bucket write. Request-path refusals are not
+// audited in this console (the plan cap's is not either), so it logs.
+func (s *Server) refuseSidecarsPaused(w http.ResponseWriter, acct *Account, app, verb string) bool {
+ if !s.sidecarsPaused(acct) {
+ return false
+ }
+ log.Printf("console: api sidecars %s %s/%s: refused, account not in CARLOS_SIDECAR_ACCOUNTS", verb, acct.Sqid, app)
+ writeJSON(w, http.StatusForbidden, map[string]any{"error": sidecarsPausedMsg, "code": sidecarsPausedCode})
+ return true
+}
diff --git a/internal/console/sidecarsgate_test.go b/internal/console/sidecarsgate_test.go
new file mode 100644
index 000000000..7f0f90dd0
--- /dev/null
+++ b/internal/console/sidecarsgate_test.go
@@ -0,0 +1,193 @@
+package console
+
+// The sidecar pause (sidecarsgate.go), through the real Server, its router
+// and the cliRoute auth stack the rest of apisidecars_test.go uses. The
+// properties, in the order they would hurt if lost: with the allowlist set,
+// an account it does not name cannot enable or create a sidecar by any
+// credential that reaches those doors, and nothing is written to the bucket
+// when it is refused; a stranger still gets the byte-identical 404, so the
+// gate never confirms an account exists; existing sidecars stay listable,
+// wakeable and deletable; a named account is unaffected; and with the
+// allowlist unset nothing changes at all.
+
+import (
+ "context"
+ "encoding/json"
+ "net/http"
+ "slices"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/provision"
+ "github.com/carlosframework/platform/internal/release"
+ "github.com/carlosframework/platform/internal/svcrole"
+)
+
+// sidecarAppKeys is every bucket key under the fixture app's prefix, sorted: the
+// refused calls must leave it exactly as it was.
+func sidecarAppKeys(t *testing.T, fx *consoleFixture) []string {
+ t.Helper()
+ keys, err := fx.rel.List(context.Background(), release.AppPrefix("bab", "titogo"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ slices.Sort(keys)
+ return keys
+}
+
+func assertSidecarsPaused(t *testing.T, what string, status int, body string) {
+ t.Helper()
+ if status != http.StatusForbidden {
+ t.Fatalf("%s = %d %s, want 403", what, status, body)
+ }
+ var got struct{ Error, Code string }
+ if err := json.Unmarshal([]byte(body), &got); err != nil {
+ t.Fatalf("%s body %q: %v", what, body, err)
+ }
+ if got.Code != "sidecars-paused" || got.Error != sidecarsPausedMsg {
+ t.Fatalf("%s body = %+v, want code sidecars-paused and the paused sentence", what, got)
+ }
+}
+
+// TestSidecarsPausedRefusesEnableAndCreate: the fixture's account (bab) is
+// not on the list, so enable and create are refused for the member, the
+// legacy app token and an instance-role service principal, and the bucket
+// is untouched.
+func TestSidecarsPausedRefusesEnableAndCreate(t *testing.T) {
+ fx, member := instanceFixture(t, "code-side-paused")
+ fx.srv.cfg.SidecarAccounts = []string{"cdef"}
+ u := sidecarsURL(fx, "bab", "titogo")
+
+ // Enable with no manifest yet: refused, and no manifest appears.
+ before := sidecarAppKeys(t, fx)
+ status, _, out := rawReq(t, http.MethodPut, sidecarEnableURL(fx, "bab", "titogo"), member, `{"channel":"beta"}`)
+ assertSidecarsPaused(t, "paused enable, no manifest", status, out)
+ if after := sidecarAppKeys(t, fx); !slices.Equal(before, after) {
+ t.Fatalf("a refused enable changed the bucket: %v -> %v", before, after)
+ }
+
+ // An operator-written manifest (store mode, not a console door) is
+ // there; enable still may not rewrite it.
+ enableSidecars(t, fx, "titogo", titoSidecarManifest())
+ manifest, _, err := fx.rel.Get(context.Background(), provision.SidecarManifestKey("bab", "titogo"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ status, _, out = rawReq(t, http.MethodPut, sidecarEnableURL(fx, "bab", "titogo"), member, `{"channel":"beta","args":["evil"]}`)
+ assertSidecarsPaused(t, "paused enable over a manifest", status, out)
+ if now, _, err := fx.rel.Get(context.Background(), provision.SidecarManifestKey("bab", "titogo")); err != nil || string(now) != string(manifest) {
+ t.Fatalf("a refused enable rewrote the manifest: %s -> %s (%v)", manifest, now, err)
+ }
+
+ // Create, by every credential the create door admits.
+ appTok, err := fx.st.NewAppToken(OpsAccountID, "titogo", "alice@keymail.dev", "signup", time.Now())
+ if err != nil {
+ t.Fatal(err)
+ }
+ svc := serviceToken(t, fx.st, OpsAccountID, "router", "titogo", svcrole.Instance)
+ before = sidecarAppKeys(t, fx)
+ for _, c := range []struct{ what, token string }{
+ {"member create", member},
+ {"app token create", appTok},
+ {"instance service principal create", svc},
+ } {
+ status, _, out := rawReq(t, http.MethodPost, u, c.token, `{"name":"bot","region":"ie"}`)
+ assertSidecarsPaused(t, c.what, status, out)
+ }
+ if after := sidecarAppKeys(t, fx); !slices.Equal(before, after) {
+ t.Fatalf("a refused create changed the bucket: %v -> %v", before, after)
+ }
+ if keys, err := fx.rel.List(context.Background(), provision.SidecarsPrefix("bab", "titogo")); err != nil || len(keys) != 0 {
+ t.Fatalf("refused creates left records %v (%v)", keys, err)
+ }
+
+ // The gate comes after membership: a stranger and an unknown account
+ // still get the byte-identical 404, never the 403 that would confirm
+ // the account exists and is paused.
+ want := canonicalNotFound(t)
+ eve := cliToken(t, fx, "eve@keymail.dev", "code-side-paused-eve")
+ for _, c := range []struct{ what, method, url, token, body string }{
+ {"non-member enable", http.MethodPut, sidecarEnableURL(fx, "bab", "titogo"), eve, `{}`},
+ {"non-member create", http.MethodPost, u, eve, `{"name":"bot","region":"ie"}`},
+ {"unknown account create", http.MethodPost, sidecarsURL(fx, "zzzz", "titogo"), member, `{"name":"bot","region":"ie"}`},
+ {"unclaimed app create", http.MethodPost, sidecarsURL(fx, "bab", "ghost-app"), member, `{"name":"bot","region":"ie"}`},
+ } {
+ status, h, out := rawReq(t, c.method, c.url, c.token, c.body)
+ assertNotFound(t, c.what, want, status, h, out)
+ }
+ // And validation still answers first for a member.
+ if status, _, out := rawReq(t, http.MethodPost, u, member, `{"name":"-nope-","region":"ie"}`); status != http.StatusBadRequest {
+ t.Fatalf("paused create with a bad name = %d %s, want the 400 first", status, out)
+ }
+}
+
+// TestSidecarsPausedLeavesExistingSidecarsAlone: a sidecar created before the
+// pause is still listed, read, woken and deleted by the paused account, by
+// the member and by the instance service principal.
+func TestSidecarsPausedLeavesExistingSidecarsAlone(t *testing.T) {
+ fx, member := instanceFixture(t, "code-side-paused-existing")
+ enableSidecars(t, fx, "titogo", titoSidecarManifest())
+ u := sidecarsURL(fx, "bab", "titogo")
+ for _, name := range []string{"bot", "bot2"} {
+ if status, _, out := rawReq(t, http.MethodPost, u, member, `{"name":"`+name+`","region":"ie"}`); status != http.StatusCreated {
+ t.Fatalf("create %s before the pause = %d %s", name, status, out)
+ }
+ }
+ fx.srv.cfg.SidecarAccounts = []string{"cdef"}
+ svc := serviceToken(t, fx.st, OpsAccountID, "router", "titogo", svcrole.Instance)
+
+ for _, c := range []struct{ what, method, url string }{
+ {"list", http.MethodGet, u},
+ {"get", http.MethodGet, u + "/bot"},
+ {"wake", http.MethodPost, u + "/bot/wake"},
+ } {
+ if status, _, out := rawReq(t, c.method, c.url, member, ""); status != http.StatusOK {
+ t.Fatalf("paused %s = %d %s, want 200", c.what, status, out)
+ }
+ }
+ for _, c := range []struct{ name, token string }{{"bot", member}, {"bot2", svc}} {
+ if status, _, out := rawReq(t, http.MethodDelete, u+"/"+c.name, c.token, ""); status != http.StatusOK {
+ t.Fatalf("paused delete of %s = %d %s, want 200", c.name, status, out)
+ }
+ if _, _, err := fx.rel.Get(context.Background(), provision.SidecarRecordKey("bab", "titogo", c.name)); err == nil {
+ t.Fatalf("the record for %s survived its delete", c.name)
+ }
+ }
+}
+
+// TestSidecarsPauseAllowlistAndUnset: an account the list names, and a
+// console with no list at all, enable and create exactly as before.
+func TestSidecarsPauseAllowlistAndUnset(t *testing.T) {
+ for _, c := range []struct {
+ what string
+ accts []string
+ }{
+ {"allowlisted", []string{"cdef", "bab"}},
+ {"unset", nil},
+ } {
+ t.Run(c.what, func(t *testing.T) {
+ fx, member := instanceFixture(t, "code-side-allow-"+c.what)
+ fx.srv.cfg.SidecarAccounts = c.accts
+ status, _, out := rawReq(t, http.MethodPut, sidecarEnableURL(fx, "bab", "titogo"), member, `{"channel":"stable"}`)
+ if status != http.StatusOK {
+ t.Fatalf("enable = %d %s, want 200", status, out)
+ }
+ // The pool and regions are operator policy the enable door never
+ // sets; put them as the operator would so create can place.
+ enableSidecars(t, fx, "titogo", titoSidecarManifest())
+ u := sidecarsURL(fx, "bab", "titogo")
+ if status, _, out := rawReq(t, http.MethodPost, u, member, `{"name":"bot","region":"ie"}`); status != http.StatusCreated {
+ t.Fatalf("member create = %d %s, want 201", status, out)
+ }
+ svc := serviceToken(t, fx.st, OpsAccountID, "router", "titogo", svcrole.Instance)
+ if status, _, out := rawReq(t, http.MethodPost, u, svc, `{"name":"bot2","region":"ie"}`); status != http.StatusCreated {
+ t.Fatalf("service principal create = %d %s, want 201", status, out)
+ }
+ for _, name := range []string{"bot", "bot2"} {
+ if _, err := provision.GetSidecarRecord(context.Background(), fx.rel, "bab", "titogo", name); err != nil {
+ t.Fatalf("record %s: %v", name, err)
+ }
+ }
+ })
+ }
+}
diff --git a/internal/provision/census_propose_internal_test.go b/internal/provision/census_propose_internal_test.go
new file mode 100644
index 000000000..19a267956
--- /dev/null
+++ b/internal/provision/census_propose_internal_test.go
@@ -0,0 +1,373 @@
+package provision
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+// proposeFixture is a small census: one legacy unit, one authority unit, one
+// envelope, sealed, and a scan that matches it.
+func proposeFixture(t *testing.T) (rootReadScan, rootReadInventory, string) {
+ t.Helper()
+ legacy := classifiedRootRead{rootReadEntry: rootReadEntry{File: "internal/a/a.go", Unit: "Wake", Kinds: "registry:Host@10,registry:DBPath@20", Digest: strings.Repeat("1", 64)}, Class: "legacy", Task: "accepted-legacy", Basis: "accepted-legacy;rev3=J1;basis=root-route"}
+ auth := classifiedRootRead{rootReadEntry: rootReadEntry{File: "internal/a/b.go", Unit: "Route", Kinds: "registry:Host@5", Digest: strings.Repeat("2", 64)}, Class: "authority", Task: "E4", Basis: "root-route"}
+ fp := rootReadLegacyFingerprint(legacy)
+ inv := rootReadInventory{
+ Entries: map[string]classifiedRootRead{"internal/a/a.go:Wake": legacy, "internal/a/b.go:Route": auth},
+ Frozen: map[string]classifiedRootRead{"internal/a/a.go:Wake": legacy},
+ Legacy: map[string]bool{fp: true},
+ Coverage: map[string]string{"internal/a": strings.Repeat("c", 64)},
+ }
+ got := rootReadScan{
+ Entries: []rootReadEntry{legacy.rootReadEntry, auth.rootReadEntry},
+ Declarations: map[string]string{"internal/a/a.go:Wake": legacy.Digest, "internal/a/b.go:Route": auth.Digest},
+ Coverage: map[string]string{"internal/a": strings.Repeat("c", 64)},
+ }
+ return got, inv, rootReadLegacySeal(inv)
+}
+
+// change edits the scan as a code change would: a new digest, and the
+// unit's read sites (kinds) as given.
+func change(got *rootReadScan, key, kinds, digest string) {
+ for i, e := range got.Entries {
+ if e.File+":"+e.Unit == key {
+ got.Entries[i].Kinds, got.Entries[i].Digest = kinds, digest
+ }
+ }
+ got.Declarations[key] = digest
+}
+
+func TestCensusClassifiesEachKindOfChange(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ // body: the same reads at moved offsets.
+ change(&got, "internal/a/a.go:Wake", "registry:Host@14,registry:DBPath@24", strings.Repeat("3", 64))
+ // reads: a kind of read the unit did not make.
+ change(&got, "internal/a/b.go:Route", "registry:Host@5,registry:Socket@9", strings.Repeat("4", 64))
+ got.Coverage["internal/a"] = strings.Repeat("d", 64)
+ got.Entries = append(got.Entries, rootReadEntry{File: "internal/a/c.go", Unit: "New", Kinds: "file:Open@1", Digest: strings.Repeat("5", 64)})
+ got.Declarations["internal/a/c.go:New"] = strings.Repeat("5", 64)
+
+ classes := map[string]string{}
+ for _, c := range classifyRootReads(got, inv, seal) {
+ classes[c.Key] = c.Class
+ }
+ want := map[string]string{
+ "internal/a/a.go:Wake": censusBody,
+ "internal/a/b.go:Route": censusReads,
+ "internal/a": censusEnvelope,
+ "internal/a/c.go:New": censusNew,
+ }
+ for k, v := range want {
+ if classes[k] != v {
+ t.Errorf("class of %s = %q, want %q (all: %v)", k, classes[k], v, classes)
+ }
+ }
+ if len(classes) != len(want) {
+ t.Errorf("classes = %v, want exactly %v", classes, want)
+ }
+}
+
+// sites: another read of a kind the unit already makes.
+func TestCensusClassifiesMoreReadsOfAKnownKindAsSites(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ change(&got, "internal/a/b.go:Route", "registry:Host@5,registry:Host@9", strings.Repeat("4", 64))
+ cs := classifyRootReads(got, inv, seal)
+ if len(cs) != 1 || cs[0].Class != censusSites {
+ t.Fatalf("changes = %+v, want one sites", cs)
+ }
+}
+
+// The card never shows less than the census: a discrepancy no unit or
+// envelope change explains is an "other" decision with the census's own
+// message (Astra review, census-approvals P2).
+func TestCensusClassifiesEveryOtherDiscrepancy(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ for name, tc := range map[string]struct {
+ edit func(*rootReadScan, *rootReadInventory) string
+ want string
+ }{
+ "a wrong seal": {
+ edit: func(*rootReadScan, *rootReadInventory) string { return strings.Repeat("e", 64) },
+ want: "changed legacy freeze",
+ },
+ "a removed declaration back in the tree": {
+ edit: func(g *rootReadScan, i *rootReadInventory) string {
+ i.Removed = map[string]string{"internal/a/z.go:Gone": "E4;census-ruling-x"}
+ g.Declarations["internal/a/z.go:Gone"] = strings.Repeat("6", 64)
+ return seal
+ },
+ want: "internal/a/z.go:Gone",
+ },
+ } {
+ t.Run(name, func(t *testing.T) {
+ g, i := got, cloneRootReadInventory(inv)
+ g.Declarations = cloneMap(got.Declarations)
+ s := tc.edit(&g, &i)
+ cs := classifyRootReads(g, i, s)
+ if len(cs) == 0 {
+ t.Fatal("classified nothing")
+ }
+ for _, c := range cs {
+ if c.Class != censusOther || !strings.Contains(c.Key, tc.want) {
+ t.Fatalf("changes = %+v, want an other carrying %q", cs, tc.want)
+ }
+ }
+ if card := rootReadCard("x", cs, nil, s, ""); strings.Contains(card, "Nothing to approve") || !strings.Contains(card, tc.want) {
+ t.Errorf("card:\n%s", card)
+ }
+ })
+ }
+}
+
+func TestCensusClassifiesARemovedUnit(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ got.Entries = got.Entries[:1]
+ delete(got.Declarations, "internal/a/b.go:Route")
+ cs := classifyRootReads(got, inv, seal)
+ if len(cs) != 1 || cs[0].Class != censusRemoved || cs[0].Key != "internal/a/b.go:Route" {
+ t.Fatalf("changes = %+v, want one removed", cs)
+ }
+}
+
+// The proposal is the whole job: loaded onto the inventory it leaves no
+// discrepancy, under the seal it reports, and every row names the ruling.
+func TestCensusProposalClosesEveryApprovableChange(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ change(&got, "internal/a/a.go:Wake", "registry:Host@14,registry:DBPath@24", strings.Repeat("3", 64))
+ change(&got, "internal/a/b.go:Route", "registry:Host@5,registry:Host@9", strings.Repeat("4", 64))
+ got.Coverage["internal/a"] = strings.Repeat("d", 64)
+ if len(censusDiscrepancies(got, inv, seal)) == 0 {
+ t.Fatal("fixture change made no discrepancy")
+ }
+ claims := map[string]string{
+ "internal/a/a.go:Wake": "logs the wake's timing; no read added",
+ "internal/a/b.go:Route": "reads the route's host again to log it",
+ }
+ p, err := proposeRootReads(got, inv, seal, "wake-timing", claims)
+ if err != nil {
+ t.Fatal(err)
+ }
+ path := filepath.Join(t.TempDir(), "overlay.tsv")
+ if err := os.WriteFile(path, []byte(p.Overlay), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ after := cloneRootReadInventory(inv)
+ if err := loadRootReadInventory(path, &after, true); err != nil {
+ t.Fatal(err)
+ }
+ if errs := censusDiscrepancies(got, after, p.Seal); len(errs) != 0 {
+ t.Fatalf("after the proposal: %v\n%s", errs, p.Overlay)
+ }
+ if rootReadLegacySeal(inv) != seal {
+ t.Error("proposing changed the inventory it was given")
+ }
+ for _, line := range strings.Split(p.Overlay, "\n") {
+ if line != "" && !strings.HasPrefix(line, "#") && !strings.HasPrefix(line, "@coverage") && !strings.Contains(line, "rev3=census-ruling-wake-timing;basis=") {
+ t.Errorf("row does not name the ruling: %s", line)
+ }
+ }
+ for _, want := range []string{"census-ruling-wake-timing", "internal/a/a.go", "body", "sites", "+registry:Host", "internal/a", seal, p.Seal, "logs the wake's timing"} {
+ if !strings.Contains(p.Card, want) {
+ t.Errorf("card lacks %q:\n%s", want, p.Card)
+ }
+ }
+}
+
+func TestCensusProposalRefuses(t *testing.T) {
+ for name, tc := range map[string]struct {
+ edit func(*rootReadScan)
+ claims map[string]string
+ want string
+ }{
+ "a change without a claim": {
+ edit: func(g *rootReadScan) {
+ change(g, "internal/a/a.go:Wake", "registry:Host@14,registry:DBPath@24", strings.Repeat("3", 64))
+ },
+ claims: map[string]string{},
+ want: "no claim",
+ },
+ "a claim for no change": {
+ edit: func(g *rootReadScan) {
+ change(g, "internal/a/a.go:Wake", "registry:Host@14,registry:DBPath@24", strings.Repeat("3", 64))
+ },
+ claims: map[string]string{"internal/a/a.go:Wake": "x", "internal/a/z.go:Typo": "y"},
+ want: "not a change",
+ },
+ "a new kind of read": {
+ edit: func(g *rootReadScan) {
+ change(g, "internal/a/b.go:Route", "registry:Host@5,file:Open@9", strings.Repeat("4", 64))
+ },
+ claims: map[string]string{"internal/a/b.go:Route": "opens a file"},
+ want: "decision",
+ },
+ "a new unit": {
+ edit: func(g *rootReadScan) {
+ g.Entries = append(g.Entries, rootReadEntry{File: "internal/a/c.go", Unit: "New", Kinds: "file:Open@1", Digest: strings.Repeat("5", 64)})
+ g.Declarations["internal/a/c.go:New"] = strings.Repeat("5", 64)
+ },
+ claims: map[string]string{"internal/a/c.go:New": "opens a file"},
+ want: "decision",
+ },
+ "a claim with a tab": {
+ edit: func(g *rootReadScan) {
+ change(g, "internal/a/a.go:Wake", "registry:Host@14,registry:DBPath@24", strings.Repeat("3", 64))
+ },
+ claims: map[string]string{"internal/a/a.go:Wake": "a\tb"},
+ want: "tab",
+ },
+ "nothing to propose": {
+ edit: func(*rootReadScan) {},
+ claims: map[string]string{},
+ want: "nothing",
+ },
+ } {
+ t.Run(name, func(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ tc.edit(&got)
+ _, err := proposeRootReads(got, inv, seal, "x", tc.claims)
+ if err == nil || !strings.Contains(err.Error(), tc.want) {
+ t.Fatalf("err = %v, want one mentioning %q", err, tc.want)
+ }
+ })
+ }
+}
+
+func TestCensusClaimsFile(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "claims")
+ if err := os.WriteFile(path, []byte("# what changed\n\ninternal/a/a.go:Wake\tlogs the timing; no read added\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ claims, err := readCensusClaims(path)
+ if err != nil || claims["internal/a/a.go:Wake"] != "logs the timing; no read added" || len(claims) != 1 {
+ t.Fatalf("claims = %v, %v", claims, err)
+ }
+ if err := os.WriteFile(path, []byte("internal/a/a.go:Wake no tab\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := readCensusClaims(path); err == nil {
+ t.Fatal("a line without a tab was accepted")
+ }
+}
+
+// writeFixtureRoot is a tree with an empty chain manifest and the notes dir.
+func writeFixtureRoot(t *testing.T) string {
+ t.Helper()
+ root := t.TempDir()
+ for _, d := range []string{"internal/provision/testdata", "docs/superpowers/notes"} {
+ if err := os.MkdirAll(filepath.Join(root, d), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if err := os.WriteFile(filepath.Join(root, rootReadChainManifest), []byte("# manifest\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ return root
+}
+
+func readFile(t *testing.T, path string) string {
+ t.Helper()
+ raw, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return string(raw)
+}
+
+func TestCensusProposalWritesAndReplacesItsOwnOverlay(t *testing.T) {
+ root := writeFixtureRoot(t)
+ first := rootReadProposal{Overlay: "# Census overlay x, " + censusProposeMark + ".\nrow1\n", Seal: strings.Repeat("a", 64), Card: "card 1"}
+ if err := writeRootReadProposal(root, "x", first); err != nil {
+ t.Fatal(err)
+ }
+ second := rootReadProposal{Overlay: "# Census overlay x, " + censusProposeMark + ".\nrow2\n", Seal: strings.Repeat("b", 64), Card: "card 2"}
+ if err := writeRootReadProposal(root, "x", second); err != nil {
+ t.Fatal(err)
+ }
+ if got, want := readFile(t, filepath.Join(root, rootReadChainManifest)), "# manifest\nplan14-root-reads-x.tsv\t"+strings.Repeat("b", 64)+"\n"; got != want {
+ t.Errorf("manifest = %q, want %q", got, want)
+ }
+ if got := readFile(t, filepath.Join(root, "internal/provision/testdata/plan14-root-reads-x.tsv")); got != second.Overlay {
+ t.Errorf("overlay = %q", got)
+ }
+ if got := readFile(t, filepath.Join(root, "docs/superpowers/notes/census-proposal-x.md")); got != "card 2" {
+ t.Errorf("card = %q", got)
+ }
+}
+
+func TestCensusProposalRefusesWhatItDidNotWrite(t *testing.T) {
+ for name, setup := range map[string]func(t *testing.T, root string){
+ "a hand-written overlay of that name": func(t *testing.T, root string) {
+ if err := os.WriteFile(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-x.tsv"), []byte("# Census overlay x (by hand)\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ },
+ "its overlay followed by another": func(t *testing.T, root string) {
+ mine := rootReadProposal{Overlay: "# " + censusProposeMark + "\n", Seal: strings.Repeat("a", 64)}
+ if err := writeRootReadProposal(root, "x", mine); err != nil {
+ t.Fatal(err)
+ }
+ if err := writeRootReadProposal(root, "y", mine); err != nil {
+ t.Fatal(err)
+ }
+ },
+ } {
+ t.Run(name, func(t *testing.T) {
+ root := writeFixtureRoot(t)
+ setup(t, root)
+ manifest := readFile(t, filepath.Join(root, rootReadChainManifest))
+ overlay, _ := os.ReadFile(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-x.tsv"))
+ card, _ := os.ReadFile(filepath.Join(root, "docs/superpowers/notes/census-proposal-x.md"))
+ p := rootReadProposal{Overlay: "# " + censusProposeMark + "\nnew\n", Seal: strings.Repeat("c", 64), Card: "card"}
+ if err := writeRootReadProposal(root, "x", p); err == nil {
+ t.Fatal("replaced an overlay it should not have")
+ }
+ if got := readFile(t, filepath.Join(root, rootReadChainManifest)); got != manifest {
+ t.Errorf("manifest changed: %q", got)
+ }
+ if got, _ := os.ReadFile(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-x.tsv")); string(got) != string(overlay) {
+ t.Errorf("overlay changed: %q", got)
+ }
+ if got, _ := os.ReadFile(filepath.Join(root, "docs/superpowers/notes/census-proposal-x.md")); string(got) != string(card) {
+ t.Errorf("card changed: %q", got)
+ }
+ })
+ }
+}
+
+// Keys may contain spaces, and a directory is a prefix of its units' keys.
+func TestCensusNamesWholeKeys(t *testing.T) {
+ for _, tc := range []struct {
+ msg, key string
+ want bool
+ }{
+ {"changed root read internal/h/k.go:type gen; review class", "internal/h/k.go:type gen", true},
+ {"changed legacy declaration internal/h/k.go:type gen", "internal/h/k.go:type gen", true},
+ {"changed root read internal/h/k.go:type general; review", "internal/h/k.go:type gen", false},
+ {"changed root read internal/a/x.go:Foo; review", "internal/a", false},
+ {"changed root source envelope internal/a", "internal/a", true},
+ {"stale root read internal/a/x.go:Foo", "internal/a/x.go:Foo", true},
+ } {
+ if got := censusNames(tc.msg, tc.key); got != tc.want {
+ t.Errorf("censusNames(%q, %q) = %v, want %v", tc.msg, tc.key, got, tc.want)
+ }
+ }
+}
+
+// A body change to a unit whose key has a space is still an approval.
+func TestCensusClassifiesAKeyWithASpace(t *testing.T) {
+ got, inv, seal := proposeFixture(t)
+ key := "internal/a/t.go:type gen"
+ e := classifiedRootRead{rootReadEntry: rootReadEntry{File: "internal/a/t.go", Unit: "type gen", Kinds: "registry:Host@3", Digest: strings.Repeat("7", 64)}, Class: "legacy", Task: "accepted-legacy", Basis: "accepted-legacy;rev3=J1;basis=root-route"}
+ inv.Entries[key], inv.Frozen[key] = e, e
+ inv.Legacy[rootReadLegacyFingerprint(e)] = true
+ seal = rootReadLegacySeal(inv)
+ got.Entries = append(got.Entries, e.rootReadEntry)
+ got.Declarations[key] = e.Digest
+ change(&got, key, "registry:Host@4", strings.Repeat("8", 64))
+ cs := classifyRootReads(got, inv, seal)
+ if len(cs) != 1 || cs[0].Class != censusBody || cs[0].Key != key {
+ t.Fatalf("changes = %+v, want one body for %q", cs, key)
+ }
+}
diff --git a/internal/provision/census_propose_test.go b/internal/provision/census_propose_test.go
new file mode 100644
index 000000000..7dd3fa823
--- /dev/null
+++ b/internal/provision/census_propose_test.go
@@ -0,0 +1,571 @@
+package provision
+
+// census-propose (#369): census rulings as approvals. The root-read census
+// (TestPlan14RootReadCensus) fails on any change to a unit it tracks or to
+// a package envelope. Most of those changes are approvals, not decisions:
+// the same reads in an edited body, or an envelope fingerprint that moved.
+// This file classifies each discrepancy, writes the overlay that closes the
+// approvable ones (rows, @legacy fingerprints, envelope re-pins, the seal,
+// the chain manifest line), and the approval card a person reads to
+// approve it. Changes that need a disposition (a new unit, a removed one,
+// the scanner's field list) are decisions: it names them and writes
+// nothing.
+//
+// make census-card print the card; writes nothing
+// make census-propose NAME=<branch> CLAIMS=<file> write the overlay and the card
+//
+// CLAIMS has one line per changed unit, "<file>:<unit>\t<what changed>".
+// Rows carry rev3=census-ruling-<NAME> from the start: approval happens
+// before the merge, so there is no PROPOSED row to flip afterwards.
+
+import (
+ "fmt"
+ "os"
+ "path/filepath"
+ "regexp"
+ "sort"
+ "strings"
+ "testing"
+)
+
+// Change classes. The first three are approvals; the rest are decisions.
+const (
+ censusBody = "body" // digest moved, the same reads (offsets aside)
+ censusSites = "sites" // more or fewer reads of kinds the unit already makes
+ censusEnvelope = "envelope" // a package's source fingerprint moved
+ censusReads = "reads" // a kind of read the unit did not make before
+ censusNew = "new" // an unclassified unit: needs a disposition
+ censusRemoved = "removed" // a unit or envelope gone: needs a removal row
+ censusOther = "other" // a census discrepancy no change above explains
+)
+
+// censusDecision: a new kind of read can make a unit's disposition wrong
+// (a registry reader that starts opening files is not the unit that was
+// classified), so it is the owner's call, as a new or removed unit is
+// (Astra review, census-approvals P1).
+func censusDecision(class string) bool {
+ return class == censusReads || class == censusNew || class == censusRemoved || class == censusOther
+}
+
+type censusChange struct {
+ Key, Class string
+ Before classifiedRootRead // the inventory's row; zero for a new unit
+ After rootReadEntry // the scan's; zero for a removed unit
+ // An envelope's fingerprints.
+ OldHash, NewHash string
+}
+
+// kindCounts is a unit's read sites without their offsets.
+func kindCounts(kinds string) map[string]int {
+ m := map[string]int{}
+ if kinds == "" {
+ return m
+ }
+ for _, k := range strings.Split(kinds, ",") {
+ name, _, _ := strings.Cut(k, "@")
+ m[name]++
+ }
+ return m
+}
+
+// kindDiff says how a unit's read sites changed: "+registry:Socket",
+// "-file:Open ×2", or "unchanged (N)".
+func kindDiff(before, after string) string {
+ b, a := kindCounts(before), kindCounts(after)
+ names := map[string]bool{}
+ for k := range b {
+ names[k] = true
+ }
+ for k := range a {
+ names[k] = true
+ }
+ var sorted []string
+ for k := range names {
+ sorted = append(sorted, k)
+ }
+ sort.Strings(sorted)
+ var parts []string
+ for _, k := range sorted {
+ d := a[k] - b[k]
+ sign := "+"
+ if d < 0 {
+ sign, d = "-", -d
+ }
+ switch {
+ case d == 1:
+ parts = append(parts, sign+k)
+ case d > 1:
+ parts = append(parts, fmt.Sprintf("%s%s ×%d", sign, k, d))
+ }
+ }
+ if len(parts) == 0 {
+ n := 0
+ for _, c := range a {
+ n += c
+ }
+ return fmt.Sprintf("unchanged (%d)", n)
+ }
+ return strings.Join(parts, ", ")
+}
+
+func sameKinds(before, after string) bool {
+ return kindDiff(before, after) == kindDiff(after, after)
+}
+
+// newKinds reports whether after reads a kind before did not.
+func newKinds(before, after string) bool {
+ b := kindCounts(before)
+ for k := range kindCounts(after) {
+ if b[k] == 0 {
+ return true
+ }
+ }
+ return false
+}
+
+// classifyRootReads names every discrepancy the census would report under
+// seal, one change per unit or envelope. A discrepancy no unit or envelope
+// change accounts for (a removed declaration back in the tree, the seal
+// itself, the scanner's field list) is an "other" decision carrying the
+// census's own message, so the card never shows less than the census does
+// (Astra review, census-approvals P2).
+func classifyRootReads(got rootReadScan, inv rootReadInventory, seal string) []censusChange {
+ var cs []censusChange
+ seen := map[string]bool{}
+ for _, g := range got.Entries {
+ key := g.File + ":" + g.Unit
+ seen[key] = true
+ w, ok := inv.Entries[key]
+ switch {
+ case !ok:
+ cs = append(cs, censusChange{Key: key, Class: censusNew, After: g})
+ case w.Kinds == g.Kinds && w.Digest == g.Digest && declarationHolds(got, inv, key):
+ case sameKinds(w.Kinds, g.Kinds):
+ cs = append(cs, censusChange{Key: key, Class: censusBody, Before: w, After: g})
+ case newKinds(w.Kinds, g.Kinds):
+ cs = append(cs, censusChange{Key: key, Class: censusReads, Before: w, After: g})
+ default:
+ cs = append(cs, censusChange{Key: key, Class: censusSites, Before: w, After: g})
+ }
+ }
+ for key, w := range inv.Entries {
+ if seen[key] {
+ continue
+ }
+ decl, declared := got.Declarations[key]
+ switch {
+ case w.Class != "legacy" && w.Kinds == "" && declared && decl == w.Digest:
+ // Declared, reads nothing, unchanged: not a discrepancy.
+ case w.Class != "legacy" && w.Kinds == "" && declared:
+ cs = append(cs, censusChange{Key: key, Class: censusBody, Before: w, After: rootReadEntry{File: w.File, Unit: w.Unit, Digest: decl}})
+ default:
+ cs = append(cs, censusChange{Key: key, Class: censusRemoved, Before: w})
+ }
+ }
+ for dir, hash := range got.Coverage {
+ if inv.Coverage[dir] != hash {
+ cs = append(cs, censusChange{Key: dir, Class: censusEnvelope, OldHash: inv.Coverage[dir], NewHash: hash})
+ }
+ }
+ for dir, hash := range inv.Coverage {
+ if _, ok := got.Coverage[dir]; !ok {
+ cs = append(cs, censusChange{Key: dir, Class: censusRemoved, OldHash: hash})
+ }
+ }
+ keys := map[string]bool{}
+ for _, c := range cs {
+ keys[c.Key] = true
+ }
+ for _, msg := range censusDiscrepancies(got, inv, seal) {
+ explained := false
+ for key := range keys {
+ if censusNames(msg, key) {
+ explained = true
+ break
+ }
+ }
+ if !explained {
+ cs = append(cs, censusChange{Key: msg, Class: censusOther})
+ }
+ }
+ sort.Slice(cs, func(i, j int) bool {
+ if cs[i].Class != cs[j].Class {
+ return cs[i].Class < cs[j].Class
+ }
+ return cs[i].Key < cs[j].Key
+ })
+ return cs
+}
+
+// censusNames reports whether a census message names key whole: keys may
+// contain spaces ("file.go:type T"), and an envelope's directory is a
+// prefix of its units' keys, so the key must be bounded by a space before
+// it and a space, a semicolon or the end after it (Astra round 2).
+func censusNames(msg, key string) bool {
+ for i := 0; ; {
+ j := strings.Index(msg[i:], " "+key)
+ if j < 0 {
+ return false
+ }
+ end := i + j + 1 + len(key)
+ if end == len(msg) || msg[end] == ' ' || msg[end] == ';' {
+ return true
+ }
+ i += j + 1
+ }
+}
+
+// declarationHolds is checkRootReadDeclarations's test for one unit.
+func declarationHolds(got rootReadScan, inv rootReadInventory, key string) bool {
+ frozen, ok := inv.Frozen[key]
+ if !ok {
+ return true
+ }
+ cur := inv.Entries[key]
+ if cur.Class == "legacy" {
+ return got.Declarations[key] == frozen.Digest && rootReadLegacyFingerprint(cur) == rootReadLegacyFingerprint(frozen)
+ }
+ return got.Declarations[key] == cur.Digest
+}
+
+type rootReadProposal struct {
+ Overlay, Seal, Card string
+}
+
+var censusName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
+
+// proposeRootReads writes the overlay that closes every change in the scan,
+// re-freezing each changed unit at the disposition it already has with its
+// claim appended, under census-ruling-<name>. It refuses decisions, a
+// change with no claim and a claim for no change, and checks its own work:
+// loaded onto a copy of inv, the overlay must leave the census clean.
+func proposeRootReads(got rootReadScan, inv rootReadInventory, seal, name string, claims map[string]string) (rootReadProposal, error) {
+ if !censusName.MatchString(name) {
+ return rootReadProposal{}, fmt.Errorf("census name %q: want lower-case letters, digits and dashes", name)
+ }
+ cs := classifyRootReads(got, inv, seal)
+ if len(cs) == 0 {
+ return rootReadProposal{}, fmt.Errorf("nothing to propose: the census is clean")
+ }
+ var decisions []string
+ units := map[string]bool{}
+ for _, c := range cs {
+ if censusDecision(c.Class) {
+ decisions = append(decisions, c.Class+" "+c.Key)
+ }
+ if c.Class == censusBody || c.Class == censusSites {
+ units[c.Key] = true
+ }
+ }
+ if len(decisions) > 0 {
+ return rootReadProposal{}, fmt.Errorf("a decision is needed, not an approval: %s; write those rows by hand and ask on #280", strings.Join(decisions, "; "))
+ }
+ for key := range units {
+ if strings.TrimSpace(claims[key]) == "" {
+ return rootReadProposal{}, fmt.Errorf("no claim for %s: say what changed in the CLAIMS file", key)
+ }
+ }
+ for key, claim := range claims {
+ if !units[key] {
+ return rootReadProposal{}, fmt.Errorf("claim for %s: not a change the census sees", key)
+ }
+ if strings.ContainsAny(claim, "\t\n\r") {
+ return rootReadProposal{}, fmt.Errorf("claim for %s: a tab or newline would break the row", key)
+ }
+ }
+
+ ruling := "census-ruling-" + name
+ var frozen, plain, cover []string
+ counts := map[string]int{}
+ for _, c := range cs {
+ counts[c.Class]++
+ if c.Class == censusEnvelope {
+ cover = append(cover, "@coverage\t"+c.Key+"\t"+c.NewHash)
+ continue
+ }
+ e := classifiedRootRead{rootReadEntry: c.After, Class: c.Before.Class, Task: c.Before.Task, Basis: c.Before.Basis + ";rev3=" + ruling + ";basis=" + strings.TrimSpace(claims[c.Key])}
+ row := strings.Join([]string{e.File, e.Unit, e.Class, e.Task, e.Basis, e.Kinds, e.Digest}, "\t")
+ if e.Class == "legacy" {
+ frozen = append(frozen, "@legacy\t"+rootReadLegacyFingerprint(e)+"\t"+row)
+ }
+ plain = append(plain, row)
+ }
+ header := []string{
+ fmt.Sprintf("# Census overlay %s, "+censusProposeMark+". %s, approved by the census owner before the merge; the approval card is docs/superpowers/notes/census-proposal-%s.md.", name, ruling, name),
+ fmt.Sprintf("# %d body, %d sites, %d envelope: each changed unit re-frozen at its existing disposition with its claim appended.", counts[censusBody], counts[censusSites], counts[censusEnvelope]),
+ }
+ overlay := strings.Join(append(append(append(header, frozen...), plain...), cover...), "\n") + "\n"
+
+ after := cloneRootReadInventory(inv)
+ f, err := os.CreateTemp("", "census-overlay-*.tsv")
+ if err != nil {
+ return rootReadProposal{}, err
+ }
+ defer os.Remove(f.Name())
+ if _, err := f.WriteString(overlay); err != nil {
+ f.Close()
+ return rootReadProposal{}, err
+ }
+ if err := f.Close(); err != nil {
+ return rootReadProposal{}, err
+ }
+ if err := loadRootReadInventory(f.Name(), &after, true); err != nil {
+ return rootReadProposal{}, fmt.Errorf("the overlay does not load: %w", err)
+ }
+ newSeal := rootReadLegacySeal(after)
+ if errs := censusDiscrepancies(got, after, newSeal); len(errs) != 0 {
+ return rootReadProposal{}, fmt.Errorf("the overlay leaves discrepancies: %s", strings.Join(errs, "; "))
+ }
+ return rootReadProposal{Overlay: overlay, Seal: newSeal, Card: rootReadCard(name, cs, claims, seal, newSeal)}, nil
+}
+
+func censusDiscrepancies(got rootReadScan, inv rootReadInventory, seal string) []string {
+ errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
+ return append(errs, checkRootReadDeclarations(got, inv)...)
+}
+
+// rootReadCard is what a person reads to approve the change. newSeal is
+// empty for a dry run (make census-card).
+func rootReadCard(name string, cs []censusChange, claims map[string]string, seal, newSeal string) string {
+ var b strings.Builder
+ fmt.Fprintf(&b, "# Census approval: %s\n\n", name)
+ var decisions []censusChange
+ var envelopes []string
+ var units []censusChange
+ for _, c := range cs {
+ switch {
+ case censusDecision(c.Class):
+ decisions = append(decisions, c)
+ case c.Class == censusEnvelope:
+ envelopes = append(envelopes, "`"+c.Key+"`")
+ default:
+ units = append(units, c)
+ }
+ }
+ switch {
+ case len(cs) == 0:
+ fmt.Fprintf(&b, "Nothing to approve: the census is clean (seal `%s`).\n", seal)
+ return b.String()
+ case len(decisions) == 0:
+ fmt.Fprintf(&b, "Ruling `census-ruling-%s`. Every change below keeps the disposition it already has, so it needs an approval, not a decision.\n\n", name)
+ default:
+ fmt.Fprintf(&b, "Ruling `census-ruling-%s`. %d change(s) need a decision (below), which census-propose cannot write.\n\n", name, len(decisions))
+ }
+ if len(units) > 0 {
+ b.WriteString("| unit | change | read sites | what changed |\n|---|---|---|---|\n")
+ for _, c := range units {
+ claim := strings.TrimSpace(claims[c.Key])
+ if claim == "" {
+ claim = "(no claim yet)"
+ }
+ fmt.Fprintf(&b, "| `%s` `%s` | %s | %s | %s |\n", c.After.File, c.After.Unit, c.Class, kindDiff(c.Before.Kinds, c.After.Kinds), claim)
+ }
+ b.WriteString("\n")
+ }
+ if len(envelopes) > 0 {
+ fmt.Fprintf(&b, "Envelopes re-pinned: %s.\n\n", strings.Join(envelopes, ", "))
+ }
+ if len(decisions) > 0 {
+ b.WriteString("Decisions (write their rows by hand and ask on #280):\n\n")
+ for _, c := range decisions {
+ detail := ""
+ switch c.Class {
+ case censusNew:
+ detail = ": reads " + kindDiff("", c.After.Kinds)
+ case censusReads:
+ detail = ": " + kindDiff(c.Before.Kinds, c.After.Kinds)
+ }
+ fmt.Fprintf(&b, "- %s `%s`%s\n", c.Class, c.Key, detail)
+ }
+ b.WriteString("\n")
+ }
+ if newSeal != "" {
+ fmt.Fprintf(&b, "Seal: `%s` → `%s`.\n\nOverlay: `internal/provision/testdata/%s`, appended to `%s`.\n", seal, newSeal, rootReadOverlayFile(name), rootReadChainManifest)
+ } else {
+ fmt.Fprintf(&b, "Seal now: `%s`. Dry run: nothing written.\n", seal)
+ }
+ return b.String()
+}
+
+// censusProposeMark is in the first line of every overlay census-propose
+// writes: a re-run replaces only a file that carries it.
+const censusProposeMark = "written by census-propose (#369)"
+
+func rootReadOverlayFile(name string) string { return "plan14-root-reads-" + name + ".tsv" }
+
+func cloneRootReadInventory(inv rootReadInventory) rootReadInventory {
+ out := rootReadInventory{Fields: inv.Fields}
+ out.Entries = cloneMap(inv.Entries)
+ out.Frozen = cloneMap(inv.Frozen)
+ out.Removed = cloneMap(inv.Removed)
+ out.Coverage = cloneMap(inv.Coverage)
+ out.Legacy = cloneMap(inv.Legacy)
+ return out
+}
+
+func cloneMap[V any](m map[string]V) map[string]V {
+ if m == nil {
+ return nil
+ }
+ out := make(map[string]V, len(m))
+ for k, v := range m {
+ out[k] = v
+ }
+ return out
+}
+
+// readCensusClaims reads a CLAIMS file: "<file>:<unit>\t<what changed>"
+// per line, blank lines and # comments ignored.
+func readCensusClaims(path string) (map[string]string, error) {
+ raw, err := os.ReadFile(path)
+ if err != nil {
+ return nil, err
+ }
+ claims := map[string]string{}
+ for i, line := range strings.Split(string(raw), "\n") {
+ line = strings.TrimRight(line, "\r")
+ if strings.TrimSpace(line) == "" || strings.HasPrefix(line, "#") {
+ continue
+ }
+ key, claim, ok := strings.Cut(line, "\t")
+ if !ok || !strings.Contains(key, ":") || strings.TrimSpace(claim) == "" {
+ return nil, fmt.Errorf("%s:%d: want <file>:<unit><TAB><what changed>", path, i+1)
+ }
+ if _, dup := claims[key]; dup {
+ return nil, fmt.Errorf("%s:%d: a second claim for %s", path, i+1, key)
+ }
+ claims[key] = claim
+ }
+ return claims, nil
+}
+
+const (
+ censusCardStart = "--- census card ---"
+ censusCardEnd = "--- end census card ---"
+)
+
+// TestPlan14RootReadCensusPropose is census-propose and census-card (see
+// the Makefile). It does nothing unless one of them asked.
+func TestPlan14RootReadCensusPropose(t *testing.T) {
+ name, dry := os.Getenv("CARLOS_CENSUS_PROPOSE"), os.Getenv("CARLOS_CENSUS_CARD") != ""
+ if name == "" && !dry {
+ t.Skip("run by make census-card and make census-propose")
+ }
+ root := censusRoot(t)
+ // A re-run (the code changed again) proposes against the chain before
+ // its own overlay, which must still be the manifest's last; nothing is
+ // changed on disk until the new proposal is built and checked.
+ before := ""
+ if !dry {
+ if !censusName.MatchString(name) {
+ t.Fatalf("census name %q: want lower-case letters, digits and dashes", name)
+ }
+ before = rootReadOverlayFile(name)
+ }
+ got, err := rootReadProduction(root)
+ if err != nil {
+ t.Fatal(err)
+ }
+ inv, seal := rootReadChainBefore(t, root, before)
+ claims := map[string]string{}
+ if path := os.Getenv("CARLOS_CENSUS_CLAIMS"); path != "" {
+ if claims, err = readCensusClaims(path); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if dry {
+ if name == "" {
+ name = "<name>"
+ }
+ fmt.Printf("%s\n%s%s\n", censusCardStart, rootReadCard(name, classifyRootReads(got, inv, seal), claims, seal, ""), censusCardEnd)
+ return
+ }
+ p, err := proposeRootReads(got, inv, seal, name, claims)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := writeRootReadProposal(root, name, p); err != nil {
+ t.Fatal(err)
+ }
+ fmt.Printf("%s\n%s%s\n", censusCardStart, p.Card, censusCardEnd)
+}
+
+// writeRootReadProposal writes a checked proposal: the overlay, its line
+// at the end of the chain manifest (replacing its own line on a re-run)
+// and the approval card. It refuses, writing nothing, to replace an
+// overlay census-propose did not write, or one that is not the manifest's
+// last.
+func writeRootReadProposal(root, name string, p rootReadProposal) error {
+ file := rootReadOverlayFile(name)
+ overlay := filepath.Join(root, "internal/provision/testdata", file)
+ if raw, err := os.ReadFile(overlay); err == nil {
+ first, _, _ := strings.Cut(string(raw), "\n")
+ if !strings.Contains(first, censusProposeMark) {
+ return fmt.Errorf("%s exists and census-propose did not write it: propose under a new name", file)
+ }
+ } else if !os.IsNotExist(err) {
+ return err
+ }
+ manifest := filepath.Join(root, rootReadChainManifest)
+ raw, err := os.ReadFile(manifest)
+ if err != nil {
+ return err
+ }
+ lines := strings.Split(strings.TrimRight(string(raw), "\n"), "\n")
+ links, err := readRootReadChainManifest(manifest)
+ if err != nil {
+ return err
+ }
+ for i, l := range links {
+ if l.File != file {
+ continue
+ }
+ if i != len(links)-1 || lines[len(lines)-1] != l.File+"\t"+l.Seal {
+ return fmt.Errorf("%s is not the last overlay in %s: propose under a new name", file, rootReadChainManifest)
+ }
+ lines = lines[:len(lines)-1]
+ }
+ lines = append(lines, file+"\t"+p.Seal)
+ note := filepath.Join(root, "docs/superpowers/notes", "census-proposal-"+name+".md")
+ // Every file is written beside its target first and renamed only once
+ // all three are on disk, the manifest last, so a failed write leaves
+ // the previous proposal whole (Astra round 2). A rename failing between
+ // the others is not rolled back (Astra round 3, declined): the files
+ // are a working tree's, the census fails on any mismatch, and a re-run
+ // recovers, since the manifest still ends in this name's line.
+ writes := []struct{ path, text string }{
+ {overlay, p.Overlay},
+ {note, p.Card},
+ {manifest, strings.Join(lines, "\n") + "\n"},
+ }
+ var temps []string
+ defer func() {
+ for _, tmp := range temps {
+ os.Remove(tmp)
+ }
+ }()
+ for _, w := range writes {
+ f, err := os.CreateTemp(filepath.Dir(w.path), "."+filepath.Base(w.path)+".*")
+ if err != nil {
+ return err
+ }
+ temps = append(temps, f.Name())
+ if _, err := f.WriteString(w.text); err != nil {
+ f.Close()
+ return err
+ }
+ if err := f.Chmod(0o644); err != nil {
+ f.Close()
+ return err
+ }
+ if err := f.Close(); err != nil {
+ return err
+ }
+ }
+ for i, w := range writes {
+ if err := os.Rename(temps[i], w.path); err != nil {
+ return err
+ }
+ }
+ return nil
+}
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go
index fc34b82d5..72cc46997 100644
--- a/internal/provision/plan14_root_reads_test.go
+++ b/internal/provision/plan14_root_reads_test.go
@@ -593,6 +593,36 @@ func TestPlan14RootReadCensus(t *testing.T) {
if err != nil {
t.Fatal(err)
}
+ inv, seal := rootReadChain(t, root)
+ errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
+ errs = append(errs, checkRootReadDeclarations(got, inv)...)
+ if len(errs) > 0 {
+ t.Fatalf("root read census: %d discrepancies\n%s\n\nmake census-card shows what changed; make census-propose NAME=<branch> CLAIMS=<file> writes the overlay and its approval card (AGENTS.md, The census)", len(errs), strings.Join(errs, "\n"))
+ }
+ counts := map[string]int{}
+ sites := 0
+ for _, e := range inv.Entries {
+ counts[e.Class]++
+ sites += len(strings.Split(e.Kinds, ","))
+ }
+ t.Logf("%d units, %d sites, %d source envelopes; classes %v", len(inv.Entries), sites, len(inv.Coverage), counts)
+}
+
+// rootReadChain loads the census inventory: the base file and every overlay
+// in order, each loaded only after its predecessor's seal is checked, then
+// the overlays the chain manifest lists (rootReadManifest). It returns the
+// inventory and the seal the last overlay leaves.
+func rootReadChain(t testing.TB, root string) (rootReadInventory, string) {
+ t.Helper()
+ return rootReadChainBefore(t, root, "")
+}
+
+// rootReadChainBefore is rootReadChain without the manifest's last overlay
+// when that overlay is the file named last (a census-propose re-run
+// proposing against the chain before its own overlay); "" loads it all.
+func rootReadChainBefore(t testing.TB, root, last string) (rootReadInventory, string) {
+ t.Helper()
+ var err error
inv := rootReadInventory{Entries: map[string]classifiedRootRead{}, Coverage: map[string]string{}}
if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads.tsv"), &inv, false); err != nil {
t.Fatal(err)
@@ -1456,18 +1486,7 @@ func TestPlan14RootReadCensus(t *testing.T) {
t.Fatal(err)
}
seal = "09e20ea4bbc534710334e632e903878656dabfb5817d4d96c6e7c56508172148"
- errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
- errs = append(errs, checkRootReadDeclarations(got, inv)...)
- if len(errs) > 0 {
- t.Fatalf("root read census: %d discrepancies\n%s", len(errs), strings.Join(errs, "\n"))
- }
- counts := map[string]int{}
- sites := 0
- for _, e := range inv.Entries {
- counts[e.Class]++
- sites += len(strings.Split(e.Kinds, ","))
- }
- t.Logf("%d units, %d sites, %d source envelopes; classes %v", len(inv.Entries), sites, len(inv.Coverage), counts)
+ return rootReadManifest(t, root, inv, seal, last)
}
func TestPlan14RootReadCensusRejectsDrift(t *testing.T) {
@@ -1726,3 +1745,60 @@ func TestPlan14ActivationToolsRemovalRequiresAbsentDeclaration(t *testing.T) {
t.Fatal("unruled removal accepted")
}
}
+
+// rootReadChainManifest lists the overlays loaded after the chain written
+// out in rootReadChain, in order, one per line: the overlay's file name
+// under testdata and the seal it leaves. census-propose appends to it
+// (rootReadPropose), so a new overlay needs no edit to this file.
+const rootReadChainManifest = "internal/provision/testdata/plan14-root-reads-chain.tsv"
+
+type rootReadChainLink struct{ File, Seal string }
+
+func readRootReadChainManifest(path string) ([]rootReadChainLink, error) {
+ raw, err := os.ReadFile(path)
+ if err != nil {
+ return nil, err
+ }
+ var links []rootReadChainLink
+ for i, line := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
+ if line == "" || strings.HasPrefix(line, "#") {
+ continue
+ }
+ col := strings.Split(line, "\t")
+ if len(col) != 2 || len(col[1]) != 64 || strings.ContainsAny(col[0], "/\\") {
+ return nil, fmt.Errorf("%s:%d: want <overlay file name>\\t<seal>", path, i+1)
+ }
+ if _, err := hex.DecodeString(col[1]); err != nil {
+ return nil, fmt.Errorf("%s:%d: seal: %v", path, i+1, err)
+ }
+ links = append(links, rootReadChainLink{col[0], col[1]})
+ }
+ return links, nil
+}
+
+// rootReadManifest loads the manifest's overlays onto inv, each after the
+// seal before it is checked, exactly as rootReadChain loads its own. A
+// non-empty last leaves that overlay out, and only when it is the last.
+func rootReadManifest(t testing.TB, root string, inv rootReadInventory, seal, last string) (rootReadInventory, string) {
+ t.Helper()
+ links, err := readRootReadChainManifest(filepath.Join(root, rootReadChainManifest))
+ if err != nil {
+ t.Fatal(err)
+ }
+ for i, l := range links {
+ if last != "" && l.File == last {
+ if i != len(links)-1 {
+ t.Fatalf("%s is not the last overlay in %s: propose under a new name", last, rootReadChainManifest)
+ }
+ break
+ }
+ if errs := checkRootReadLegacy(inv, seal); len(errs) != 0 {
+ t.Fatalf("before overlay %s: %v", l.File, errs)
+ }
+ if err := loadRootReadInventory(filepath.Join(root, "internal/provision/testdata", l.File), &inv, true); err != nil {
+ t.Fatal(err)
+ }
+ seal = l.Seal
+ }
+ return inv, seal
+}
diff --git a/internal/provision/plan9_b8_census_test.go b/internal/provision/plan9_b8_census_test.go
index 525edfb2c..ec0176aee 100644
--- a/internal/provision/plan9_b8_census_test.go
+++ b/internal/provision/plan9_b8_census_test.go
@@ -366,7 +366,7 @@ func TestPlan9B8RecordWriteCensus(t *testing.T) {
if len(got) < 100 {
t.Fatalf("census found only %d units under %s; the walk is broken", len(got), root)
}
- inventory := dbRegionForward3WriteCensusInventory()
+ inventory := secSidecarCreateGateWriteCensusInventory()
problems := append(censusDiff(inventory, got), censusPolicy(inventory)...)
if len(problems) > 0 {
t.Fatalf("record-write census rejected %d site(s); update the explicit inventory only after reviewing the source:\n %s", len(problems), strings.Join(problems, "\n "))
diff --git a/internal/provision/sec_sidecar_create_gate_census_test.go b/internal/provision/sec_sidecar_create_gate_census_test.go
new file mode 100644
index 000000000..c4ecffc32
--- /dev/null
+++ b/internal/provision/sec_sidecar_create_gate_census_test.go
@@ -0,0 +1,27 @@
+package provision
+
+// Record-write census entries for the sidecar pause (sec-sidecar-create-gate),
+// layered on dbRegionForward3WriteCensusInventory (main's last at 65ca4aca).
+// handleAPISidecarCreate gains one early return, refuseSidecarsPaused
+// (internal/console/sidecarsgate.go), after the shape check and before the
+// plan cap, the leftover rail and both write arms. Its write is unchanged:
+// the same CreateSidecarRecord create-only write, or the host-claim create.
+// No write site is added; the refusal writes nothing.
+func secSidecarCreateGateWriteCensusInventory() []censusEntry {
+ entries := append([]censusEntry(nil), dbRegionForward3WriteCensusInventory()...)
+ replacements := []censusEntry{
+ {"internal/console/apisidecars.go", "Server.handleAPISidecarCreate", "record-writer", "key:CreateSidecarRecord", "72f4d44028fbc46ed2ed1adca6154f385b62e143f47e597a49b6e28570b2ddab"},
+ }
+ for _, r := range replacements {
+ found := false
+ for i, e := range entries {
+ if e.File == r.File && e.Unit == r.Unit {
+ entries[i], found = r, true
+ }
+ }
+ if !found {
+ entries = append(entries, r)
+ }
+ }
+ return entries
+}
diff --git a/internal/provision/testdata/plan14-root-reads-chain.tsv b/internal/provision/testdata/plan14-root-reads-chain.tsv
new file mode 100644
index 000000000..b55f07f47
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-chain.tsv
@@ -0,0 +1,6 @@
+# Census chain manifest: overlays loaded after rootReadChain's own, in order.
+# One per line: <overlay file name under testdata>\t<the seal it leaves>.
+# make census-propose appends here; nothing else edits this file.
+plan14-root-reads-sec-host-imdsv2-module.tsv 945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a
+plan14-root-reads-sec-deps-dos-bumps.tsv 945934f5978ace14dc5a9bae0ed69dfabf29b994504fc843c03d4a633ecd6c6a
+plan14-root-reads-sec-sidecar-create-gate.tsv 0ca3baafa26f59c0b7d8c6d6a39237d074afb71f885b7fbd50debebab5bf89f2
diff --git a/internal/provision/testdata/plan14-root-reads-sec-deps-dos-bumps.tsv b/internal/provision/testdata/plan14-root-reads-sec-deps-dos-bumps.tsv
new file mode 100644
index 000000000..02c33bd1e
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-sec-deps-dos-bumps.tsv
@@ -0,0 +1,3 @@
+# Census overlay sec-deps-dos-bumps, written by census-propose (#369). census-ruling-sec-deps-dos-bumps, approved by the census owner before the merge; the approval card is docs/superpowers/notes/census-proposal-sec-deps-dos-bumps.md.
+# 0 body, 0 sites, 1 envelope: each changed unit re-frozen at its existing disposition with its claim appended.
+@coverage . ff7e60e8734238290264ba3b27931d9477811d9a37921a205b1b7244a610bef0
diff --git a/internal/provision/testdata/plan14-root-reads-sec-host-imdsv2-module.tsv b/internal/provision/testdata/plan14-root-reads-sec-host-imdsv2-module.tsv
new file mode 100644
index 000000000..c4ac824f6
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-sec-host-imdsv2-module.tsv
@@ -0,0 +1,5 @@
+# Census overlay sec-host-imdsv2-module, written by census-propose (#369). census-ruling-sec-host-imdsv2-module, approved by the census owner before the merge; the approval card is docs/superpowers/notes/census-proposal-sec-host-imdsv2-module.md.
+# 1 body, 0 sites, 1 envelope: each changed unit re-frozen at its existing disposition with its claim appended.
+@legacy bf41de79166910448a12a9ba5e5568bc7365193b909d5ebbe942facce8ecb95c infra/modules/region-host/main.tf external legacy accepted-legacy accepted-legacy;rev3=census-ruling-console-invite-delivery;basis=tofu on an operator's machine with the operator's AWS credentials (no box unit; no binary embeds the file): main's module with one optional aws_iam_role_policy, host_console_mail, count 0 unless console_mail_from is set; it grants the host role ses:SendEmail on the one sending identity, the carlos-bab configuration set and its tenants, conditioned on ses:FromAddress equal to console_mail_from; no read is added and nothing on a box reads the file;rev3=census-ruling-sec-host-imdsv2-module;basis=aws_instance.host gains metadata_options (IMDSv2 required, hop limit 1, endpoint enabled), matching scanner-host; tofu configuration only, run on an operator's machine; no read, file or record write is added; applies in place on a running instance with no restart file:external@0 8ec89e75ca0a0d2ddde74bffe226adb5a5935249fd41590bcce1623c5201ab47
+infra/modules/region-host/main.tf external legacy accepted-legacy accepted-legacy;rev3=census-ruling-console-invite-delivery;basis=tofu on an operator's machine with the operator's AWS credentials (no box unit; no binary embeds the file): main's module with one optional aws_iam_role_policy, host_console_mail, count 0 unless console_mail_from is set; it grants the host role ses:SendEmail on the one sending identity, the carlos-bab configuration set and its tenants, conditioned on ses:FromAddress equal to console_mail_from; no read is added and nothing on a box reads the file;rev3=census-ruling-sec-host-imdsv2-module;basis=aws_instance.host gains metadata_options (IMDSv2 required, hop limit 1, endpoint enabled), matching scanner-host; tofu configuration only, run on an operator's machine; no read, file or record write is added; applies in place on a running instance with no restart file:external@0 8ec89e75ca0a0d2ddde74bffe226adb5a5935249fd41590bcce1623c5201ab47
+@coverage infra/modules/region-host 5f1b1bbc02603a54e136ff004e2495bef2b0f22054e058b5930a539749b1b558
diff --git a/internal/provision/testdata/plan14-root-reads-sec-sidecar-create-gate.tsv b/internal/provision/testdata/plan14-root-reads-sec-sidecar-create-gate.tsv
new file mode 100644
index 000000000..ff9e1e70c
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-sec-sidecar-create-gate.tsv
@@ -0,0 +1,6 @@
+# Census overlay sec-sidecar-create-gate, written by census-propose (#369). census-ruling-sec-sidecar-create-gate, approved by the census owner before the merge; the approval card is docs/superpowers/notes/census-proposal-sec-sidecar-create-gate.md.
+# 1 body, 0 sites, 2 envelope: each changed unit re-frozen at its existing disposition with its claim appended.
+@legacy dd17558eb17031da9bd9c517c68fd97ac5456969222be618e4b24c74128813fd cmd/console/main.go serve legacy accepted-legacy accepted-legacy;rev3=PROPOSED-scan-demand-revival;basis=Console service wires the optional callback signer and starts the claim-guarded queue worker. The same three data path literals move; it does not add a privileged reader.;rev3=PROPOSED-email-signin;identity=the console (cmd/console serve under console.service, user carlos-console);basis=passes CARLOS_CONSOLE_NAME to NewServer as Config.ConsoleName, the name the console's emails use; the same three /data literals at the same offsets; no read added;rev3=PROPOSED-s3-change-index;identity=the console (cmd/console serve under console.service, user carlos-console);basis=arms the record view when CARLOS_CHANGE_INDEX=read (armChangeIndex) for the run and seed sweeps and starts its backstop loop; the same three /data literals at the same offsets; no read added;rev3=census-ruling-db-region-forward-2;identity=the console (cmd/console serve under console.service, user carlos-console);basis=parses CARLOS_DB_FORWARD (dbForward) into Config.DBForward and builds its http.Server with console.NewHTTPServer, the same two timeouts; the same three /data literals at the same offsets; no read added;rev3=census-ruling-sec-sidecar-create-gate;basis=serve now parses CARLOS_SIDECAR_ACCOUNTS (sidecarAccounts, a pure env parse with log.Fatalf on a bad or empty-list value) before opening anything and passes it into console.Config.SidecarAccounts. No read was added: the same 3 root read sites. data-literal@4319,data-literal@5420,data-literal@5657 1f1aa5c40845b20922a7740ec15254a47ff5856e1ef14252a189e1af56ddd406
+cmd/console/main.go serve legacy accepted-legacy accepted-legacy;rev3=PROPOSED-scan-demand-revival;basis=Console service wires the optional callback signer and starts the claim-guarded queue worker. The same three data path literals move; it does not add a privileged reader.;rev3=PROPOSED-email-signin;identity=the console (cmd/console serve under console.service, user carlos-console);basis=passes CARLOS_CONSOLE_NAME to NewServer as Config.ConsoleName, the name the console's emails use; the same three /data literals at the same offsets; no read added;rev3=PROPOSED-s3-change-index;identity=the console (cmd/console serve under console.service, user carlos-console);basis=arms the record view when CARLOS_CHANGE_INDEX=read (armChangeIndex) for the run and seed sweeps and starts its backstop loop; the same three /data literals at the same offsets; no read added;rev3=census-ruling-db-region-forward-2;identity=the console (cmd/console serve under console.service, user carlos-console);basis=parses CARLOS_DB_FORWARD (dbForward) into Config.DBForward and builds its http.Server with console.NewHTTPServer, the same two timeouts; the same three /data literals at the same offsets; no read added;rev3=census-ruling-sec-sidecar-create-gate;basis=serve now parses CARLOS_SIDECAR_ACCOUNTS (sidecarAccounts, a pure env parse with log.Fatalf on a bad or empty-list value) before opening anything and passes it into console.Config.SidecarAccounts. No read was added: the same 3 root read sites. data-literal@4319,data-literal@5420,data-literal@5657 1f1aa5c40845b20922a7740ec15254a47ff5856e1ef14252a189e1af56ddd406
+@coverage cmd/console 4865ecf0f8de501d62bcfbd10cc9a4244c3e5c5ee53c6c2ee476b5dba7fd36b2
+@coverage internal/console 3ab92802ced1e88b3b841f122fa64b201d576e9dfd5db7662da0028caf7c78ec