amadan

carlos / platform Public

box publish-record: census ruling R1-R4 and Astra's review

Census ruling box-publish-record (platform-1e):
- R1: the verb refuses euid 0 first, before any store, IMDS or host read
  (a uid seam in boxPublishRun). The overlay header records "root
  unreachable: verb refuses euid 0 (R1)"; still envelope-only, `.` and
  internal/host re-pinned, seal fc584e11.
- R2: systemd-run -p EnvironmentFile=/etc/carlos/host.env -p User=carlos
  is the one supported way to run it, in the usage, the help and cli.md.
- R3: a test publishes through the loop's path and expects the verb to
  find the same evidence and write nothing.
- R4: boxRecordStore.Put stays nonrecord; the guard test runs on the
  S3Store and asserts zero requests for delete, list, overwrite and any
  other key. PROPOSED-box-publish-record becomes
  census-ruling-box-publish-record.

Astra (CHANGES NEEDED on ea68e8a6):
- P2: the verb reads the record already there through the guard and
  accepts it only with exactly this box's identity document and signature,
  version 1, this box, no other fields (sameEvidence); only written_at may
  differ. Anything else is refused and named. A record arriving after
  that read is held to the same check. publishBoxRecord is unchanged.
- P2: an S3 bucket is required: CARLOS_DEPLOYMENT_BUCKET unset, or a store
  that is not S3Store, is refused, since a deployment directory cannot
  refuse an overwrite.
- P3: failures before any write say nothing was written; once a create
  reached the bucket, the line says it may have been written. A create
  retried into 412 with this run's bytes is reported as published.
- Coverage: a group box publishes under its group reference, the loop's
  label.

Copy deck updated (not committed: copy-review/ is excluded locally).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude 3165a3bf310031f98cd95e0b0c07749e502e15a9 parent ea68e8a
9 files changed, +596 −236
  • docs/site/cli.md +4 −4
  • docs/superpowers/notes/census-proposal-box-publish-record.md +95 −72
  • internal/host/boxpublish.go +157 −44
  • internal/host/boxpublish_test.go +313 −98
  • internal/provision/box_publish_record_census_test.go +7 −5
  • internal/provision/plan14_root_reads_test.go +4 −3
  • internal/provision/testdata/plan14-root-reads-box-publish-record.tsv +6 −5
  • system.go +3 −3
  • system_roles_test.go +7 −2

CI passed — run details

CI log 
=== gate ===
go: downloading github.com/aws/aws-sdk-go-v2 v1.44.0
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/aws/aws-sdk-go-v2/config v1.32.6
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/route53 v1.64.1
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/aws/aws-sdk-go-v2/service/acm v1.45.0
go: downloading github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.68.0
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading github.com/aws/smithy-go v1.28.1
go: downloading golang.org/x/crypto v0.53.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sns v1.43.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ssm v1.74.0
go: downloading golang.org/x/sys v0.46.0
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.19.6
go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.105.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.45.6
go: downloading github.com/aws/aws-sdk-go-v2/service/iam v1.58.2
go: downloading github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.37.0
go: downloading github.com/superfly/ltx v0.5.1
go: downloading github.com/oschwald/maxminddb-golang v1.13.1
go: downloading github.com/klauspost/compress v1.18.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sesv2 v1.66.6
go: downloading golang.org/x/net v0.56.0
go: downloading github.com/pion/logging v0.2.4
go: downloading github.com/pion/stun/v3 v3.0.1
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading github.com/aws/aws-sdk-go-v2/service/costexplorer v1.67.5
go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16
go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4
go: downloading github.com/aws/aws-sdk-go-v2/service/signin v1.0.4
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.30.8
go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12
go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40
go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.38
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31
go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/pion/transport/v4 v4.0.1
go: downloading github.com/pion/randutil v0.1.0
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading golang.org/x/sync v0.21.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/prometheus/common v0.45.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0
go: downloading golang.org/x/text v0.38.0
go: downloading modernc.org/libc v1.74.1
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.011s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
go: downloading golang.org/x/tools v0.47.0
go: downloading golang.org/x/mod v0.37.0
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	329.063s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	2.919s
ok  	github.com/carlosframework/platform/cmd/moveprobe	93.287s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.701s
ok  	github.com/carlosframework/platform/internal/activator	250.302s
ok  	github.com/carlosframework/platform/internal/agentsock	0.015s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.020s
ok  	github.com/carlosframework/platform/internal/appconfig	2.483s
ok  	github.com/carlosframework/platform/internal/apperr	0.074s
ok  	github.com/carlosframework/platform/internal/applog	0.142s
ok  	github.com/carlosframework/platform/internal/awsinv	0.003s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.004s
ok  	github.com/carlosframework/platform/internal/boxid	0.565s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.008s
ok  	github.com/carlosframework/platform/internal/caps	0.016s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.701s
ok  	github.com/carlosframework/platform/internal/changelog	0.321s
ok  	github.com/carlosframework/platform/internal/claimactivation	0.025s
ok  	github.com/carlosframework/platform/internal/clamd	0.055s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	411.400s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.833s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.010s
ok  	github.com/carlosframework/platform/internal/dbverify	0.005s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.003s
ok  	github.com/carlosframework/platform/internal/docsite	0.006s
ok  	github.com/carlosframework/platform/internal/domains	2.409s
ok  	github.com/carlosframework/platform/internal/doorkey	0.007s
ok  	github.com/carlosframework/platform/internal/economics	0.005s
ok  	github.com/carlosframework/platform/internal/edge	45.216s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.009s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.134s
ok  	github.com/carlosframework/platform/internal/gate	1.655s
ok  	github.com/carlosframework/platform/internal/genselect	0.015s
ok  	github.com/carlosframework/platform/internal/geo	0.010s
ok  	github.com/carlosframework/platform/internal/geoip	0.035s
ok  	github.com/carlosframework/platform/internal/historyreadback	0.154s
ok  	github.com/carlosframework/platform/internal/host	183.892s
ok  	github.com/carlosframework/platform/internal/hostactors	6.318s
ok  	github.com/carlosframework/platform/internal/hostactors/awsmembership	0.005s
ok  	github.com/carlosframework/platform/internal/hostbox	2.746s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.593s
ok  	github.com/carlosframework/platform/internal/hostenv	0.004s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.006s
ok  	github.com/carlosframework/platform/internal/hostexec	5.264s
ok  	github.com/carlosframework/platform/internal/hostfence	0.029s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.906s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.012s
ok  	github.com/carlosframework/platform/internal/hostguard	1.035s
ok  	github.com/carlosframework/platform/internal/hostops	1.645s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.004s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.002s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.511s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.160s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.002s
ok  	github.com/carlosframework/platform/internal/infraview	0.019s
ok  	github.com/carlosframework/platform/internal/instancectl	5.272s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.757s
ok  	github.com/carlosframework/platform/internal/instancefs	0.023s
ok  	github.com/carlosframework/platform/internal/instancekey	0.022s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.024s
ok  	github.com/carlosframework/platform/internal/layout	1.568s
ok  	github.com/carlosframework/platform/internal/lbhealth	0.005s
ok  	github.com/carlosframework/platform/internal/lease	9.852s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.006s
ok  	github.com/carlosframework/platform/internal/ledger	0.013s
ok  	github.com/carlosframework/platform/internal/mail	0.209s
ok  	github.com/carlosframework/platform/internal/maintenance	0.079s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.636s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.092s
ok  	github.com/carlosframework/platform/internal/metrics	0.007s
ok  	github.com/carlosframework/platform/internal/moves	21.679s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.802s
ok  	github.com/carlosframework/platform/internal/objstore	0.199s
ok  	github.com/carlosframework/platform/internal/ops	0.189s
ok  	github.com/carlosframework/platform/internal/park	0.219s
ok  	github.com/carlosframework/platform/internal/privatefile	0.004s
ok  	github.com/carlosframework/platform/internal/provision	96.258s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.007s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	5.764s
ok  	github.com/carlosframework/platform/internal/relay	7.251s
ok  	github.com/carlosframework/platform/internal/release	23.484s
ok  	github.com/carlosframework/platform/internal/rollout	3.669s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.004s
ok  	github.com/carlosframework/platform/internal/rootaws	0.007s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.058s
ok  	github.com/carlosframework/platform/internal/rootstate	0.026s
ok  	github.com/carlosframework/platform/internal/runs	0.585s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.498s
ok  	github.com/carlosframework/platform/internal/seeds	0.014s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	11.983s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.631s
ok  	github.com/carlosframework/platform/internal/status	0.059s
ok  	github.com/carlosframework/platform/internal/steering	0.008s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.251s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.004s
ok  	github.com/carlosframework/platform/internal/usage	0.005s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.597s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 49348770 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: downloading https://github.com/minio/minio/releases/download/RELEASE.2025-09-07T16-13-09Z/minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z
  minio: downloaded, digest verified
  mc: downloading https://github.com/minio/mc/releases/download/RELEASE.2025-08-13T08-35-41Z/mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z
  mc: downloaded, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.024s
=== RUN   TestT4StorageProbeMinio
2026/10/18 00:57:01 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DF79BB0F325182, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.023s
minio contract: PASS
diff --git a/docs/site/cli.md b/docs/site/cli.md
index c637893a6..c2b3f688c 100644
--- a/docs/site/cli.md
+++ b/docs/site/cli.md
@@ -2266,14 +2266,14 @@ Ops account owners and `CARLOS_SYSTEM_OPERATORS`.
#### A box's identity record
-carlos system box publish-record publishes the box's identity record to the deployment bucket, at control/boxes/<box>.json, where the host claim's enrollment reads it. The record is the box's EC2 instance identity document and AWS's signature over it, so AWS vouches for the box, not the box itself or you. The agent publishes it by itself only in a region with HA on. Anywhere else, run this once on the box, as root or the agent user, with the box's host.env loaded.
-
-It only ever creates the record. If the box's own record is already there, it says so and writes nothing. If a record for another instance holds the box's name, it refuses and changes nothing. A fleet box publishes no record, so it refuses there too.
+carlos system box publish-record publishes the box's identity record to the deployment bucket, at control/boxes/<box>.json, where the host claim's enrollment reads it. The record is the box's EC2 instance identity document and AWS's signature over it, so AWS vouches for the box, not the box itself or you. The agent publishes it by itself only in a region with HA on. Anywhere else, run this once on the box as the agent user, with the box's host.env loaded. This is the only supported way to run it:
```sh
-carlos system box publish-record
+systemd-run --wait --pipe -p EnvironmentFile=/etc/carlos/host.env -p User=carlos /opt/carlos/carlos system box publish-record
```
+It refuses to run as root, on a fleet box, and on a box with no S3 deployment bucket. It only ever creates the record, and only an S3 bucket can refuse to overwrite one. If a record is already there with exactly the evidence this box would publish, it says so and writes nothing. If the record there differs in any other way, for example it is for another instance, it refuses, says what differs, and changes nothing.
+
#### Boxes a region's group launches
In a region with HA on, the boxes its Auto Scaling group launches set themselves up with no hand steps. A box's start-up runs two commands as root before it starts the agent.
diff --git a/docs/superpowers/notes/census-proposal-box-publish-record.md b/docs/superpowers/notes/census-proposal-box-publish-record.md
index ed08629bc..c9f5e072c 100644
--- a/docs/superpowers/notes/census-proposal-box-publish-record.md
+++ b/docs/superpowers/notes/census-proposal-box-publish-record.md
@@ -1,26 +1,67 @@
-# Census proposal: carlos system box publish-record
+# Census: carlos system box publish-record
-Status: PROPOSED-box-publish-record. platform-1e rules on every row here, as
-the host claim's owner.
+Status: RULED as census-ruling-box-publish-record (platform-1e, R1-R4), on
+ea68e8a6. This note records the ruling and the fix round that applied it.
+platform-1e rules as the host claim's owner.
Branch `box-publish-record`, cut from main at b1118925 (region HA 2b).
+## The ruling
+
+- **R1.** The verb refuses euid 0 before any store, IMDS or host read. Root
+ cannot reach `publishBoxRecord` through it. The overlay header records
+ "root unreachable: verb refuses euid 0 (R1)" rather than a unit row.
+- **R2.** The one supported way to run it:
+ `systemd-run --wait --pipe -p EnvironmentFile=/etc/carlos/host.env -p User=carlos /opt/carlos/carlos system box publish-record`.
+- **R3.** No region HA gate. Run after the loop has published, the verb is a
+ no-op: it finds exactly the same evidence and writes nothing.
+- **R4.** `boxRecordStore.Put` is `nonrecord`, on condition that
+ `boxRecordStore` is the only store the verb hands `publishBoxRecord`. The
+ store-guard test asserts zero S3 requests for a delete, a list, an
+ overwrite and any other key.
+
## What changed
`carlos system box publish-record` makes one `publishBoxRecord` attempt on a
box, by hand, without `boxRecordLoop`'s `CARLOS_REGION_HA` gate. The dev claim
trial needs records at `control/boxes/<box>.json` for leo-climbing
-(carlos-dev-ie-b) and sophie-knitting (carlos-dev-ie), and region HA stays off
-on dev.
+(carlos-dev-ie-b) and sophie-knitting (carlos-dev-ie). Region HA stays off on
+dev.
- `internal/host/boxpublish.go` (new): `RunSystemBoxPublishRecord`,
- `publishRecordOnce`, `boxRecordStore` and the verb's lines.
+ `boxPublishRun`, `sameEvidence`, `boxRecordStore` and the verb's lines.
- `system_roles.go`: `runSystemBoxVerb` hands `box publish-record` to the
host verb before `roleArgs`.
- `system.go`: `systemUsage` gains the synopsis and description lines.
- `internal/host/boxrecord.go` is not edited. `publishBoxRecord`,
`boxRecordLoop` and its gate are byte for byte main's.
+The verb's checks, in order:
+
+1. The effective uid is not 0 (R1).
+2. The box is not a fleet box (`shouldSkipPublisherElection`).
+3. `CARLOS_DEPLOYMENT_BUCKET` is set. A deployment directory is refused: its
+ check-then-rename cannot refuse an overwrite, and the verb never opens one.
+4. The box's label is `metricsBoxLabel`, the loop's, and it is a valid box
+ reference. This is the first host read (`/etc/machine-id`, or IMDS for a
+ group box).
+5. The store is opened with `openDeployStoreEnv` and must be a
+ `release.S3Store`.
+6. The record under the key is read through `boxRecordStore`, then this
+ box's record is read from IMDS once.
+7. If a record is there, `sameEvidence` decides. The envelope must be
+ version 1, this box, no other fields and nothing after it. The identity
+ document and signature must be byte for byte this box's. Only `written_at`
+ may differ. Equal is a no-op success. Anything else is refused, the
+ mismatch is named, and nothing is written.
+8. If nothing is there, `publishBoxRecord` runs through `boxRecordStore` with
+ the record already read. Its outcome is reported by what reached the
+ bucket. A failure before any create says nothing was written. A failure
+ after a create reached the bucket says the write may have happened. A
+ create answered 412 whose stored bytes are exactly this run's (a retried
+ create) is reported as published. A record that arrived after step 6 is
+ held to `sameEvidence` too.
+
## Root-read census (TestPlan14RootReadCensus)
Overlay `internal/provision/testdata/plan14-root-reads-box-publish-record.tsv`,
@@ -29,86 +70,68 @@ loaded after `plan14-root-reads-region-ha-2b.tsv`.
The scan finds no read site in any new or changed unit, so no root-read unit
is added or changed. The only discrepancies were the two source envelopes.
-| Unit | Proposed disposition | Reason |
+| Unit | Disposition | Reason |
|---|---|---|
-| `@coverage .` | re-pin to `c4e6efe2…a179` | `system.go` (usage lines) and `system_roles.go` (dispatch) changed. Neither adds a read. |
-| `@coverage internal/host` | re-pin to `03d139a8…9327` | `boxpublish.go` is new. Its units have no read site of their own. |
+| `@coverage .` | re-pinned to `d123dce7…5402` | `system.go` (usage lines) and `system_roles.go` (dispatch) changed. Neither adds a read. |
+| `@coverage internal/host` | re-pinned to `ef673c8c…60ac` | `boxpublish.go` is new. Its units have no read site of their own, and root is refused before any read (R1). |
| legacy seal | unchanged (`fc584e11…1b5d`) | The overlay adds no `@legacy` row. |
-What the verb reads when run as root, all through existing units whose
-dispositions do not change:
-
-- the box's label: `metricsBoxLabel`, and `readMachineID` for `/etc/machine-id`;
-- the deployment store: `openDeployStoreEnv`, so `CARLOS_DEPLOYMENT_BUCKET`
- through `release.NewS3Store` and `rootaws`;
-- IMDS: `readIdentityRecord` and `imdsGetRaw`, both accepted-legacy (J4,
- root-input);
-- one bucket key, `control/boxes/<box>.json`: `publishBoxRecord` through
- `boxid.Load` and `boxid.Publish`.
-
-Points for the ruling:
-
-1. **A new root entry point with no new read unit.** Root can now reach
- `publishBoxRecord`, which until now ran only in the agent (as the agent
- user) behind region HA. The census sees no unit to classify because the
- reads are in units it already classifies. The verb may also run as the
- agent user. It does not need root: the box's instance role and IMDS answer
- either user.
-2. **Nothing it reads grants authority.** The record is evidence for
- enrollment, signed by AWS. The only decision the verb takes from what it
- reads is whether to write. It writes only when nothing is under the key,
- and refuses when a record for another instance is there. The bucket
- enforces create-only twice over: `If-None-Match: *` on the PUT, and the dev
- bucket's deny-only `BoxRecordsAreCreateOnly` and
- `BoxRecordsAreNeverDeleted` statements.
-3. **It refuses on fleet boxes** (`shouldSkipPublisherElection`), before any
- store or IMDS read, the way `boxRecordLoop` returns on one.
-
-Proposed: envelope-only, at main's dispositions, seal unchanged, on the
-condition that `boxRecordStore` stays the only store the verb hands
-`publishBoxRecord` (point 4 below).
-
## Record-write census (TestPlan9B8RecordWriteCensus)
The inventory is re-pinned by `internal/provision/box_publish_record_census_test.go`
-(`plan14BoxPublishRecordWriteCensusInventory`, which the test now uses). It
+(`plan14BoxPublishRecordWriteCensusInventory`, which the test uses). It
appends one entry to `plan14ActivationToolsWriteCensusInventory`.
-| Unit | Kinds | Proposed class | Reason |
+| Unit | Kinds | Class | Reason |
|---|---|---|---|
-| `internal/host/boxpublish.go:boxRecordStore.Put` | `def:Put,sel:Put` | `nonrecord` | It writes `control/boxes/<box>.json`, a box identity record, never an instance or sidecar record. It touches no record-key constructor or record path literal. It admits only a create-only Put on its one key. |
+| `internal/host/boxpublish.go:boxRecordStore.Put` | `def:Put,sel:Put` | `nonrecord` (R4) | It writes `control/boxes/<box>.json`, a box identity record, never an instance or sidecar record. It touches no record-key constructor or record path literal. It admits only a create-only Put on its one key. |
-Digest `e5628ec8d3c7e22f88ab877209f578bfbc49f8d7a2d444da42f649625d643ea8`.
+Digest `866978d6c6786a351bbeaf6f9e5382df0b3fd08cbb47aed0c01738f7c5120d7a`.
-4. **The new caller of the control/boxes write.** Before this branch the one
- writer of `control/boxes/` was `boxid.Publish`, reached only from
- `publishBoxRecord` in `boxRecordLoop`. The verb is a second caller of the
- same chain, not a second writer: `boxid.Publish` (`nonrecord`, `sel:Put`)
- is unchanged. What is new is `boxRecordStore`, which wraps the store the
- chain writes through. It refuses before the bucket any Put on another key,
- any Put with an etag (replace-only), and every `Delete`, `List` and
- `ListDirs`. `Get` is admitted on the one key only. The census sees its
- `Put` method by name, so it is the unit to classify.
+Before this branch, the one writer of `control/boxes/` was `boxid.Publish`,
+reached only from `publishBoxRecord` in `boxRecordLoop`. The verb is a second
+caller of the same chain, not a second writer: `boxid.Publish` (`nonrecord`,
+`sel:Put`) is unchanged. `boxRecordStore` wraps the store the chain writes
+through, and refuses before the bucket any Put on another key, any Put with an
+etag, and every `Delete`, `List` and `ListDirs`. `Get` is admitted on the one
+key only.
## Tests behind the claims
`internal/host/boxpublish_test.go` runs the real `S3Store` (through
-`openDeployStoreEnv`) against `leasetest`'s S3 server, behind a fake of the dev
-bucket's policy, which denies a PUT under `control/boxes/` without
-`If-None-Match: *` and any DELETE there:
-
-- a fresh box: the record is written at the right key with the right fields,
- by one create-only PUT;
-- the same instance already recorded: a no-op success, no PUT;
-- a different instance recorded: refused, no PUT, bytes unchanged;
-- a different record created between the verb's read and its write: the
- bucket's 412 stops the PUT and nothing is overwritten;
-- a fleet box: refused with no bucket request and no IMDS read;
-- `boxRecordStore` refuses everything but read and create on its key, before
- the inner store;
-- `boxRecordLoop` still returns at once with region HA off, unparseable or on
- a fleet box, and sends nothing.
+`openDeployStoreEnv`) against `leasetest`'s S3 server. The server sits behind
+a fake of the dev bucket's policy, which denies a PUT under `control/boxes/`
+without `If-None-Match: *` and any DELETE there.
+
+- Fresh box: the record is written at the right key with the right fields,
+ by one create-only PUT.
+- After the loop has published (R3): already in place, no PUT, bytes
+ unchanged.
+- Any other evidence is refused, named, with no PUT and bytes unchanged:
+ - another instance;
+ - the same instance with another document;
+ - a different or missing signature;
+ - the wrong box;
+ - version 2;
+ - an unknown field;
+ - trailing data;
+ - not JSON.
+- A record arriving between the verb's read and `publishBoxRecord`'s is held
+ to the same evidence.
+- A different record created just before the PUT: the bucket's 412 stops it
+ and nothing is overwritten.
+- A create committed but answered 500: the SDK retries, gets 412, and the
+ verb reports published.
+- A create committed but answered 400: the verb says the write may have
+ happened, and a second run reports it in place.
+- Root (R1), a fleet box, and no S3 bucket (a directory set, or a non-S3
+ store) are each refused before the machine id, the store or IMDS is
+ touched, with zero bucket requests. The directory is never written.
+- A group box publishes under its group reference, the loop's label.
+- `boxRecordStore` (R4): zero S3 requests for a delete, a list, an
+ overwrite, another box's key and any other key.
+- `boxRecordLoop` still returns at once with region HA off, unparseable, or
+ on a fleet box, and sends nothing.
`system_roles_test.go` runs `carlos system box publish-record` from
-`runSystem` in a child process on a fleet box, and checks the refusal and the
-exit status.
+`runSystem` in a child process, and checks the refusal and the exit status.
diff --git a/internal/host/boxpublish.go b/internal/host/boxpublish.go
index 5603c50b0..6ab302ce9 100644
--- a/internal/host/boxpublish.go
+++ b/internal/host/boxpublish.go
@@ -1,10 +1,13 @@
package host
import (
+ "bytes"
"context"
+ "encoding/json"
"errors"
"flag"
"fmt"
+ "io"
"net/http"
"os"
"time"
@@ -15,38 +18,53 @@ import (
)
// carlos system box publish-record: one publishBoxRecord attempt, by hand,
-// without the CARLOS_REGION_HA gate boxRecordLoop has.
+// without the CARLOS_REGION_HA gate boxRecordLoop has (census ruling
+// box-publish-record, R1-R4).
//
-// The host claim's enrollment step needs a box's signed identity record at
+// The host claim's enrollment needs a box's signed identity record at
// control/boxes/<box>.json, and the agent writes it only when region HA is
-// on. This verb lets an operator standing on a box publish it once, with
-// the box's own AWS credentials and IMDS: AWS signs the identity document,
-// not the box and not the operator. It runs as root or as the agent user,
-// with host.env loaded, the way the agent's environment is.
+// on. This verb lets an operator publish it once, with the box's own AWS
+// credentials and IMDS: AWS signs the identity document, not the box and
+// not the operator.
//
-// It changes nothing the loop does. It refuses on a fleet box, as the loop
-// skips one. It hands publishBoxRecord a store that can read and create
-// this box's one record key and nothing else (boxRecordStore), so it can
-// never delete or overwrite: the dev bucket's box_records policy
+// It runs as the agent user and refuses root before any store, IMDS or
+// host read (R1); the one supported way to run it is systemd-run with
+// User=carlos and host.env as its EnvironmentFile (R2). It refuses on a
+// fleet box, as the loop skips one, and anywhere without an S3 deployment
+// bucket: only S3's If-None-Match makes the write create-only, and a
+// deployment directory's check-then-rename does not.
+//
+// It reads the record already under the box's key first. One that is
+// there counts as this box's only if it carries exactly the evidence the
+// box would publish now (sameEvidence): only written_at may differ.
+// Anything else is refused and left alone. With nothing there, it calls
+// publishBoxRecord through boxRecordStore, which can read and create this
+// box's one record key and nothing else (R4), so it can never delete or
+// overwrite: the dev bucket's box_records policy
// (BoxRecordsAreCreateOnly, BoxRecordsAreNeverDeleted) denies both anyway.
+const boxPublishRecordCommand = "systemd-run --wait --pipe -p EnvironmentFile=/etc/carlos/host.env -p User=carlos /opt/carlos/carlos system box publish-record"
+
const boxPublishRecordUsage = "usage: carlos system box publish-record\n\n" +
- "Publish this box's AWS-signed identity record to the deployment bucket, once.\n" +
- "Run it on the box, as root or the agent user, with host.env loaded. It never replaces or deletes a record.\n"
+ "Publish this box's AWS-signed identity record to the deployment bucket, once. It never replaces or deletes a record.\n" +
+ "Run it on the box as the agent user, with host.env loaded:\n\n" +
+ " " + boxPublishRecordCommand + "\n"
// errBoxRecordStore is boxRecordStore refusing anything but reading and
// creating this box's own record. publishBoxRecord never asks for more, so
// a user only sees it if that changes.
var errBoxRecordStore = errors.New("publish-record only reads and creates this box's own record")
-// boxRecordStore is the only store publish-record gives publishBoxRecord:
-// Get on this box's record key, and Put on it create-only (ifMatch "",
-// which S3Store sends as If-None-Match: *). Everything else is refused
-// before it reaches the bucket. wrote is whether this run's create landed.
+// boxRecordStore is the only store publish-record gives publishBoxRecord
+// (R4): Get on this box's record key, and Put on it create-only (ifMatch
+// "", which S3Store sends as If-None-Match: *). Everything else is
+// refused before it reaches the bucket. attempted is whether a create
+// reached the bucket, and wrote whether one was answered as done.
type boxRecordStore struct {
- inner release.Store
- key string
- wrote bool
+ inner release.Store
+ key string
+ attempted bool
+ wrote bool
}
func (s *boxRecordStore) Get(ctx context.Context, key string) ([]byte, string, error) {
@@ -60,6 +78,7 @@ func (s *boxRecordStore) Put(ctx context.Context, key string, body []byte, ifMat
if key != s.key || ifMatch != "" {
return "", errBoxRecordStore
}
+ s.attempted = true
etag, err := s.inner.Put(ctx, key, body, "")
if err == nil {
s.wrote = true
@@ -87,10 +106,16 @@ func boxRecordInPlaceLine(box, instance string) string {
}
const (
+ boxRecordRootRefusal = "carlos system box publish-record runs as the agent user, not root. run it with systemd-run -p User=carlos."
boxRecordFleetRefusal = "this is a fleet box. fleet boxes publish no identity record, so nothing was written"
+ boxRecordNoBucket = "this box has no deployment bucket (CARLOS_DEPLOYMENT_BUCKET is not set). the record is written only to an S3 bucket, which refuses to overwrite it, so nothing was written"
boxRecordNoNameRefusal = "this box's name could not be read, so nothing was written"
)
+func boxRecordBadName(box string) error {
+ return fmt.Errorf("this box's name %q is not a valid box reference, so nothing was written", box)
+}
+
func boxRecordStoreFailure(err error) error {
return fmt.Errorf("the deployment bucket could not be opened: %v. nothing was written", err)
}
@@ -99,49 +124,136 @@ func boxRecordRefusal(err error) error {
return fmt.Errorf("refused: %v. nothing was changed", err)
}
+func boxRecordMismatch(box, what string) error {
+ return fmt.Errorf("refused: the record already under %s does not match what this box would publish: %s. nothing was changed", box, what)
+}
+
+// boxRecordRetryable is a failure before any write reached the bucket.
func boxRecordRetryable(box string, err error) error {
return fmt.Errorf("the identity record for %s was not published: %v. nothing was written, so it is safe to run again", box, err)
}
-// publishRecordOnce is the verb: the fleet refusal, the box's own label
-// (metricsBoxLabel, as boxRecordLoop takes it), its deployment store, and
-// one publishBoxRecord through boxRecordStore. read builds the record from
-// IMDS. It returns the line to print, or the refusal.
-func publishRecordOnce(ctx context.Context, getenv func(string) string, machineID string,
- open func(context.Context, func(string) string) (release.Store, error),
- read func(box string) (boxid.Record, error)) (string, error) {
- if shouldSkipPublisherElection(getenv) {
+// boxRecordUncertain is a failure once a create reached the bucket: S3 may
+// have committed it. A second run only ever creates, and reports what it
+// finds.
+func boxRecordUncertain(box string, err error) error {
+ return fmt.Errorf("the identity record for %s may or may not have been written: %v. it is safe to run again, which only ever creates the record and says what is there", box, err)
+}
+
+// sameEvidence says how raw, the record already under the box's key,
+// differs from mine, the record this box would publish now, or "" when it
+// carries the same evidence. The envelope must be what enrollment reads
+// (version 1, this box, no other fields, nothing after it), and the
+// identity document and signature must be byte for byte this box's. Only
+// written_at may differ.
+func sameEvidence(raw []byte, mine boxid.Record) string {
+ dec := json.NewDecoder(bytes.NewReader(raw))
+ dec.DisallowUnknownFields()
+ var held boxid.Record
+ if err := dec.Decode(&held); err != nil {
+ return "it cannot be read as an identity record"
+ }
+ if err := dec.Decode(&struct{}{}); !errors.Is(err, io.EOF) {
+ return "it cannot be read as an identity record"
+ }
+ switch {
+ case held.V != 1:
+ return fmt.Sprintf("its version is %d, not 1", held.V)
+ case held.Box != mine.Box:
+ return fmt.Sprintf("it is for box %q", held.Box)
+ case held.IdentityDocument != mine.IdentityDocument:
+ theirs, terr := recordInstanceID(held)
+ own, oerr := recordInstanceID(mine)
+ if terr == nil && oerr == nil && theirs != own {
+ return fmt.Sprintf("it is signed for instance %s, and this box is %s", theirs, own)
+ }
+ return "its identity document differs from the one AWS serves this box"
+ case held.IdentitySignature != mine.IdentitySignature:
+ return "its identity signature differs from the one AWS serves this box"
+ }
+ return ""
+}
+
+// boxPublishRun is one run of the verb, with its seams: the environment,
+// the effective uid (R1), the machine id (a host read, made only after the
+// uid check), the deployment store and the IMDS read.
+type boxPublishRun struct {
+ getenv func(string) string
+ geteuid func() int
+ machineID func() string
+ open func(context.Context, func(string) string) (release.Store, error)
+ read func(box string) (boxid.Record, error)
+}
+
+// publish is the verb. It returns the line to print, or the refusal.
+func (r boxPublishRun) publish(ctx context.Context) (string, error) {
+ if r.geteuid() == 0 {
+ return "", errors.New(boxRecordRootRefusal)
+ }
+ if shouldSkipPublisherElection(r.getenv) {
return "", errors.New(boxRecordFleetRefusal)
}
- box := metricsBoxLabel(getenv, machineID)
+ if r.getenv("CARLOS_DEPLOYMENT_BUCKET") == "" {
+ return "", errors.New(boxRecordNoBucket)
+ }
+ box := metricsBoxLabel(r.getenv, r.machineID())
if box == "" {
return "", errors.New(boxRecordNoNameRefusal)
}
- inner, err := open(ctx, getenv)
+ if !boxid.ValidBox(box) {
+ return "", boxRecordBadName(box)
+ }
+ opened, err := r.open(ctx, r.getenv)
if err != nil {
return "", boxRecordStoreFailure(err)
}
- store := &boxRecordStore{inner: inner, key: boxid.Key(box)}
- var mine boxid.Record
- done, err := publishBoxRecord(ctx, store, box, func() (boxid.Record, error) {
- rec, err := read(box)
- mine = rec
- return rec, err
- })
+ if _, ok := opened.(*release.S3Store); !ok {
+ return "", errors.New(boxRecordNoBucket)
+ }
+ store := &boxRecordStore{inner: opened, key: boxid.Key(box)}
+
+ existing, _, gerr := store.Get(ctx, store.key)
+ if gerr != nil && !errors.Is(gerr, os.ErrNotExist) {
+ return "", boxRecordRetryable(box, gerr)
+ }
+ mine, err := r.read(box)
+ if err != nil {
+ return "", boxRecordRetryable(box, err)
+ }
+ instance, ierr := recordInstanceID(mine)
+ if ierr != nil {
+ instance = "unknown"
+ }
+ if gerr == nil {
+ if what := sameEvidence(existing, mine); what != "" {
+ return "", boxRecordMismatch(box, what)
+ }
+ return boxRecordInPlaceLine(box, instance), nil
+ }
+
+ done, err := publishBoxRecord(ctx, store, box, func() (boxid.Record, error) { return mine, nil })
switch {
case err != nil && done:
return "", boxRecordRefusal(err)
+ case err != nil && store.attempted:
+ return "", boxRecordUncertain(box, err)
case err != nil:
return "", boxRecordRetryable(box, err)
+ case store.attempted:
+ // Either this create landed, or a retry of it was answered 412
+ // and boxid.Publish found exactly these bytes, written_at
+ // included: this run's record either way.
+ return boxRecordPublishedLine(box, instance), nil
}
- instance, err := recordInstanceID(mine)
+ // No create was attempted: a record arrived after the read above and
+ // publishBoxRecord took it as this instance's. It is held to the same
+ // evidence as one that was there from the start.
+ raw, _, err := store.Get(ctx, store.key)
if err != nil {
- // publishBoxRecord has already read this instance id to compare
- // it, or written the record whole; it cannot fail here.
- instance = "unknown"
+ return "", boxRecordRetryable(box, err)
}
- if store.wrote {
- return boxRecordPublishedLine(box, instance), nil
+ if what := sameEvidence(raw, mine); what != "" {
+ return "", boxRecordMismatch(box, what)
}
return boxRecordInPlaceLine(box, instance), nil
}
@@ -156,8 +268,9 @@ func RunSystemBoxPublishRecord(args []string) {
os.Exit(2)
}
client := &http.Client{Timeout: 2 * time.Second}
- line, err := publishRecordOnce(context.Background(), os.Getenv, readMachineID(), openDeployStoreEnv,
- func(box string) (boxid.Record, error) { return readIdentityRecord(client, imdsBase, box, time.Now()) })
+ run := boxPublishRun{getenv: os.Getenv, geteuid: os.Geteuid, machineID: readMachineID, open: openDeployStoreEnv,
+ read: func(box string) (boxid.Record, error) { return readIdentityRecord(client, imdsBase, box, time.Now()) }}
+ line, err := run.publish(context.Background())
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
diff --git a/internal/host/boxpublish_test.go b/internal/host/boxpublish_test.go
index fed33e953..e61442657 100644
--- a/internal/host/boxpublish_test.go
+++ b/internal/host/boxpublish_test.go
@@ -1,6 +1,7 @@
package host
import (
+ "bytes"
"context"
"encoding/base64"
"encoding/json"
@@ -14,6 +15,7 @@ import (
"time"
"github.com/carlosframework/platform/internal/boxid"
+ "github.com/carlosframework/platform/internal/hostactors"
"github.com/carlosframework/platform/internal/lease/leasetest"
"github.com/carlosframework/platform/internal/release"
"github.com/carlosframework/platform/internal/status"
@@ -29,6 +31,7 @@ const (
publishMachineID = "0123456789abcdef0123456789abcdef"
publishBucket = "deploy"
publishDoc = "{\n \"accountId\" : \"111122223333\",\n \"region\" : \"eu-west-1\",\n \"instanceId\" : \"i-0abc\"\n}\n"
+ publishAgentUID = 999
)
var publishSig = base64.StdEncoding.EncodeToString([]byte(strings.Repeat("s", 256)))
@@ -40,14 +43,23 @@ type s3Request struct{ Method, Key, IfNoneMatch, IfMatch string }
// server, behind the dev bucket's deny-only box_records policy. Under
// control/boxes/ a PUT without If-None-Match: * is AccessDenied
// (BoxRecordsAreCreateOnly) and so is any DELETE
-// (BoxRecordsAreNeverDeleted). Every request is recorded. beforePut, when
-// set, runs once before the first PUT reaches the server: a racing writer.
+// (BoxRecordsAreNeverDeleted). Every request is recorded.
+//
+// Hooks, each used once: afterGet runs after the first GET has been
+// answered (a writer arriving between the verb's read and
+// publishBoxRecord's); beforePut runs before the first PUT reaches the
+// server (a writer arriving just before the create). commitThenFail lists
+// statuses for successive PUTs: each such PUT is applied and then answered
+// with that status instead, as when S3 commits a write whose answer is
+// lost.
type boxRecordsBucket struct {
model *leasetest.VersionedBucket
mu sync.Mutex
seen []s3Request
- beforePut func()
+ afterGet func()
+ beforePut func()
+ commitThenFail []int
}
func (b *boxRecordsBucket) requests() []s3Request {
@@ -62,27 +74,45 @@ func (b *boxRecordsBucket) handler() http.Handler {
_, key, _ := strings.Cut(strings.TrimPrefix(r.URL.Path, "/"), "/")
b.mu.Lock()
b.seen = append(b.seen, s3Request{r.Method, key, r.Header.Get("If-None-Match"), r.Header.Get("If-Match")})
- race := b.beforePut
- if r.Method == http.MethodPut {
- b.beforePut = nil
+ var afterGet, beforePut func()
+ fail := 0
+ switch r.Method {
+ case http.MethodGet:
+ afterGet, b.afterGet = b.afterGet, nil
+ case http.MethodPut:
+ beforePut, b.beforePut = b.beforePut, nil
+ if len(b.commitThenFail) > 0 {
+ fail, b.commitThenFail = b.commitThenFail[0], b.commitThenFail[1:]
+ }
}
b.mu.Unlock()
if strings.HasPrefix(key, boxid.Prefix) {
- denied := r.Method == http.MethodDelete || (r.Method == http.MethodPut && r.Header.Get("If-None-Match") != "*")
- if denied {
- w.Header().Set("Content-Type", "application/xml")
- w.WriteHeader(http.StatusForbidden)
- _, _ = w.Write([]byte("<Error><Code>AccessDenied</Code><Message>Access Denied</Message></Error>"))
+ if r.Method == http.MethodDelete || (r.Method == http.MethodPut && r.Header.Get("If-None-Match") != "*") {
+ s3Error(w, http.StatusForbidden, "AccessDenied")
return
}
}
- if r.Method == http.MethodPut && race != nil {
- race()
+ if beforePut != nil {
+ beforePut()
+ }
+ if fail != 0 {
+ next.ServeHTTP(httptest.NewRecorder(), r)
+ s3Error(w, fail, http.StatusText(fail))
+ return
}
next.ServeHTTP(w, r)
+ if afterGet != nil {
+ afterGet()
+ }
})
}
+func s3Error(w http.ResponseWriter, status int, code string) {
+ w.Header().Set("Content-Type", "application/xml")
+ w.WriteHeader(status)
+ _, _ = w.Write([]byte("<Error><Code>" + strings.ReplaceAll(code, " ", "") + "</Code><Message>" + code + "</Message></Error>"))
+}
+
// publishFixture points the process's AWS and CARLOS_* environment at a
// fresh fake bucket, as host.env would on a box.
func publishFixture(t *testing.T) *boxRecordsBucket {
@@ -104,27 +134,43 @@ func publishFixture(t *testing.T) *boxRecordsBucket {
return b
}
-func publishIMDS() func(string) (boxid.Record, error) {
+func publishIMDSRead(written time.Time) func(string) (boxid.Record, error) {
client := imdsClient(&fakeIMDS{doc: publishDoc, sig: publishSig})
return func(box string) (boxid.Record, error) {
- return readIdentityRecord(client, "http://imds.test", box, recordNow)
+ return readIdentityRecord(client, "http://imds.test", box, written)
}
}
+// publishRun is the verb's production wiring with the test's seams: the
+// agent user's uid, the fixed machine id and the fake IMDS.
+func publishRun() boxPublishRun {
+ return boxPublishRun{getenv: os.Getenv, geteuid: func() int { return publishAgentUID },
+ machineID: func() string { return publishMachineID }, open: openDeployStoreEnv, read: publishIMDSRead(recordNow)}
+}
+
func runPublish(t *testing.T) (string, error) {
t.Helper()
- return publishRecordOnce(context.Background(), os.Getenv, publishMachineID, openDeployStoreEnv, publishIMDS())
+ return publishRun().publish(context.Background())
}
-// assertCreateOnly: no DELETE, and every PUT under control/boxes/ was
-// create-only. Overwrites would also have been denied by the fake's
+// thisBoxRecord is the record this box would publish, written at writtenAt.
+func thisBoxRecord(t *testing.T, writtenAt string) boxid.Record {
+ t.Helper()
+ return boxid.Record{V: 1, Box: status.Label(publishMachineID), IdentityDocument: base64.StdEncoding.EncodeToString([]byte(publishDoc)),
+ IdentitySignature: publishSig, WrittenAt: writtenAt}
+}
+
+// assertCreateOnly: no DELETE, no request off this box's key, and every
+// PUT create-only. Overwrites would also have been denied by the fake's
// policy, but the verb must never even ask.
func assertCreateOnly(t *testing.T, reqs []s3Request) (puts int) {
t.Helper()
+ key := boxid.Key(status.Label(publishMachineID))
for _, r := range reqs {
+ if r.Key != key {
+ t.Errorf("%s %s: publish-record touched another key", r.Method, r.Key)
+ }
switch r.Method {
- case http.MethodDelete:
- t.Errorf("publish-record sent DELETE %s", r.Key)
case http.MethodPut:
puts++
if r.IfNoneMatch != "*" || r.IfMatch != "" {
@@ -169,20 +215,25 @@ func TestPublishRecordWritesAFreshBoxsRecord(t *testing.T) {
if puts := assertCreateOnly(t, b.requests()); puts != 1 {
t.Fatalf("%d PUTs, want one", puts)
}
- for _, r := range b.requests() {
- if r.Key != "control/boxes/"+box+".json" {
- t.Errorf("%s %s: publish-record touched another key", r.Method, r.Key)
- }
- }
}
-func TestPublishRecordIsANoOpWhenThisInstanceIsRecorded(t *testing.T) {
+// R3: there is no region HA gate. Run after the loop has published (its
+// publishBoxRecord, store, label and IMDS read, an hour earlier), the
+// verb finds exactly its own evidence and changes nothing.
+func TestPublishRecordAfterTheLoopIsANoOp(t *testing.T) {
b := publishFixture(t)
- box := status.Label(publishMachineID)
- held := identityRecord("i-0abc", "2026-10-01T00:00:00Z")
- held.Box = box
- before, _ := json.Marshal(held)
- b.model.Put(publishBucket, boxid.Key(box), before)
+ ctx := context.Background()
+ store, err := openDeployStoreEnv(ctx, os.Getenv)
+ if err != nil {
+ t.Fatal(err)
+ }
+ box := metricsBoxLabel(os.Getenv, publishMachineID)
+ loopRead := publishIMDSRead(recordNow.Add(-time.Hour))
+ if done, err := publishBoxRecord(ctx, store, box, func() (boxid.Record, error) { return loopRead(box) }); !done || err != nil {
+ t.Fatalf("the loop's publish: done=%v err=%v", done, err)
+ }
+ before, _ := b.model.Current(publishBucket, boxid.Key(box))
+ seenBefore := len(b.requests())
line, err := runPublish(t)
if err != nil {
t.Fatal(err)
@@ -191,44 +242,109 @@ func TestPublishRecordIsANoOpWhenThisInstanceIsRecorded(t *testing.T) {
t.Fatalf("line = %q\nwant %q", line, want)
}
if after, _ := b.model.Current(publishBucket, boxid.Key(box)); string(after) != string(before) {
- t.Fatalf("the record was rewritten:\n%s\n%s", before, after)
+ t.Fatalf("the loop's record was rewritten:\n%s\n%s", before, after)
}
- if puts := assertCreateOnly(t, b.requests()); puts != 0 {
- t.Fatalf("%d PUTs for a record already in place, want none", puts)
+ if puts := assertCreateOnly(t, b.requests()[seenBefore:]); puts != 0 {
+ t.Fatalf("%d PUTs, want none", puts)
}
}
-func TestPublishRecordRefusesARecordForAnotherInstance(t *testing.T) {
- b := publishFixture(t)
+// Astra P2 (1): a record already under the key counts as this box's only
+// with exactly its evidence. Anything else is refused, named, and left
+// alone.
+func TestPublishRecordRefusesAnyOtherEvidence(t *testing.T) {
box := status.Label(publishMachineID)
- held := identityRecord("i-0zzz", "2026-10-01T00:00:00Z")
- held.Box = box
- before, _ := json.Marshal(held)
- b.model.Put(publishBucket, boxid.Key(box), before)
- line, err := runPublish(t)
- if err == nil {
- t.Fatalf("published over another instance's record: %q", line)
- }
- if want := "refused: the record already under " + box + " is signed for instance i-0zzz, this box is i-0abc. nothing was changed"; err.Error() != want {
- t.Fatalf("err = %q\nwant %q", err, want)
- }
- if after, _ := b.model.Current(publishBucket, boxid.Key(box)); string(after) != string(before) {
- t.Fatalf("the record was changed:\n%s\n%s", before, after)
+ otherDoc := base64.StdEncoding.EncodeToString([]byte(strings.Replace(publishDoc, "\n}", ",\n \"pendingTime\" : \"2026-10-07T00:00:00Z\"\n}", 1)))
+ zzzDoc := base64.StdEncoding.EncodeToString([]byte(strings.Replace(publishDoc, "i-0abc", "i-0zzz", 1)))
+ prefix := "refused: the record already under " + box + " does not match what this box would publish: "
+ cases := map[string]struct {
+ edit func(*boxid.Record)
+ raw func([]byte) []byte
+ want string
+ }{
+ "another instance": {edit: func(r *boxid.Record) { r.IdentityDocument = zzzDoc }, want: "it is signed for instance i-0zzz, and this box is i-0abc"},
+ "same instance, other doc": {edit: func(r *boxid.Record) { r.IdentityDocument = otherDoc }, want: "its identity document differs from the one AWS serves this box"},
+ "different signature": {edit: func(r *boxid.Record) { r.IdentitySignature = "c2ln" }, want: "its identity signature differs from the one AWS serves this box"},
+ "missing signature": {edit: func(r *boxid.Record) { r.IdentitySignature = "" }, want: "its identity signature differs from the one AWS serves this box"},
+ "wrong box": {edit: func(r *boxid.Record) { r.Box = "bert-sawing" }, want: `it is for box "bert-sawing"`},
+ "version 2": {edit: func(r *boxid.Record) { r.V = 2 }, want: "its version is 2, not 1"},
+ "unknown field": {raw: func(b []byte) []byte { return append(bytes.TrimSuffix(b, []byte("}")), []byte(`,"extra":1}`)...) }, want: "it cannot be read as an identity record"},
+ "trailing data": {raw: func(b []byte) []byte { return append(b, []byte(`{}`)...) }, want: "it cannot be read as an identity record"},
+ "not json": {raw: func([]byte) []byte { return []byte("not json") }, want: "it cannot be read as an identity record"},
+ }
+ for name, tc := range cases {
+ t.Run(name, func(t *testing.T) {
+ b := publishFixture(t)
+ held := thisBoxRecord(t, "2026-10-01T00:00:00Z")
+ if tc.edit != nil {
+ tc.edit(&held)
+ }
+ before := mustJSON(t, held)
+ if tc.raw != nil {
+ before = tc.raw(before)
+ }
+ b.model.Put(publishBucket, boxid.Key(box), before)
+ line, err := runPublish(t)
+ if want := prefix + tc.want + ". nothing was changed"; err == nil || err.Error() != want {
+ t.Fatalf("line=%q err=%v\nwant %q", line, err, want)
+ }
+ if after, _ := b.model.Current(publishBucket, boxid.Key(box)); string(after) != string(before) {
+ t.Fatalf("the record was changed:\n%s\n%s", before, after)
+ }
+ if puts := assertCreateOnly(t, b.requests()); puts != 0 {
+ t.Fatalf("%d PUTs, want none", puts)
+ }
+ })
}
- if puts := assertCreateOnly(t, b.requests()); puts != 0 {
- t.Fatalf("%d PUTs, want none", puts)
+}
+
+// A record that arrives after the verb's read and before
+// publishBoxRecord's is held to the same evidence: publishBoxRecord alone
+// would take any record naming this instance.
+func TestPublishRecordHoldsALateRecordToTheSameEvidence(t *testing.T) {
+ box := status.Label(publishMachineID)
+ same := thisBoxRecord(t, "2026-10-01T00:00:00Z")
+ forged := same
+ forged.IdentitySignature = "c2ln"
+ for name, tc := range map[string]struct {
+ held boxid.Record
+ want string
+ fail bool
+ }{
+ "same evidence": {same, "the identity record for " + box + " (instance i-0abc) is already in place at control/boxes/" + box + ".json. nothing was written", false},
+ "same instance, other signature": {forged, "refused: the record already under " + box + " does not match what this box would publish: its identity signature differs from the one AWS serves this box. nothing was changed", true},
+ } {
+ t.Run(name, func(t *testing.T) {
+ b := publishFixture(t)
+ late := mustJSON(t, tc.held)
+ b.afterGet = func() { b.model.Put(publishBucket, boxid.Key(box), late) }
+ line, err := runPublish(t)
+ got := line
+ if err != nil {
+ got = err.Error()
+ }
+ if got != tc.want || (err != nil) != tc.fail {
+ t.Fatalf("line=%q err=%v\nwant %q", line, err, tc.want)
+ }
+ if after, _ := b.model.Current(publishBucket, boxid.Key(box)); string(after) != string(late) {
+ t.Fatalf("the late record was changed:\n%s", after)
+ }
+ if puts := assertCreateOnly(t, b.requests()); puts != 0 {
+ t.Fatalf("%d PUTs, want none", puts)
+ }
+ })
}
}
// The conditional create is the bucket's: a record another instance
-// creates after the verb's read and before its write is not overwritten,
-// because the fake answers the If-None-Match: * PUT with 412.
+// creates just before the verb's write is not overwritten, because the
+// fake answers the If-None-Match: * PUT with 412.
func TestPublishRecordLosesARaceWithoutOverwriting(t *testing.T) {
b := publishFixture(t)
box := status.Label(publishMachineID)
- held := identityRecord("i-0zzz", "2026-10-01T00:00:00Z")
- held.Box = box
- racer, _ := json.Marshal(held)
+ held := thisBoxRecord(t, "2026-10-01T00:00:00Z")
+ held.IdentityDocument = base64.StdEncoding.EncodeToString([]byte(strings.Replace(publishDoc, "i-0abc", "i-0zzz", 1)))
+ racer := mustJSON(t, held)
b.beforePut = func() { b.model.Put(publishBucket, boxid.Key(box), racer) }
want := "refused: another record already holds " + box + ": boxid: " + box + " already has a different record: store: compare-and-swap conflict. nothing was changed"
if line, err := runPublish(t); err == nil || err.Error() != want {
@@ -242,28 +358,56 @@ func TestPublishRecordLosesARaceWithoutOverwriting(t *testing.T) {
}
}
-func TestPublishRecordRefusesAFleetBox(t *testing.T) {
+// Astra P3: S3 commits the create but its answer is lost (500). The SDK
+// retries, the retry is answered 412, and boxid.Publish finds exactly
+// this run's bytes. That is this run's record: published, not already in
+// place, although the store never saw a successful Put.
+func TestPublishRecordReportsARetriedCreateAsPublished(t *testing.T) {
b := publishFixture(t)
- t.Setenv("CARLOS_FLEET", "acme")
- t.Setenv("CARLOS_FLEET_BOX", "kass-1")
- imdsRead := false
- _, err := publishRecordOnce(context.Background(), os.Getenv, publishMachineID, openDeployStoreEnv,
- func(string) (boxid.Record, error) { imdsRead = true; return boxid.Record{}, nil })
- if err == nil || err.Error() != boxRecordFleetRefusal {
- t.Fatalf("err = %v, want %q", err, boxRecordFleetRefusal)
+ box := status.Label(publishMachineID)
+ b.commitThenFail = []int{http.StatusInternalServerError}
+ line, err := runPublish(t)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if want := "published the identity record for " + box + " (instance i-0abc) to control/boxes/" + box + ".json"; line != want {
+ t.Fatalf("line = %q\nwant %q", line, want)
}
- if n := len(b.requests()); n != 0 || imdsRead {
- t.Fatalf("a fleet box reached the bucket (%d requests) or IMDS (%v)", n, imdsRead)
+ if got, _ := b.model.Current(publishBucket, boxid.Key(box)); string(got) != string(mustJSON(t, thisBoxRecord(t, "2026-10-06T09:00:00Z"))) {
+ t.Fatalf("stored %s", got)
+ }
+ if puts := assertCreateOnly(t, b.requests()); puts != 2 {
+ t.Fatalf("%d PUTs, want the create and its retry", puts)
}
}
-// A failure publishBoxRecord would retry is one attempt here, reported as
-// safe to run again, and writes nothing.
+// Astra P3: once a create has reached the bucket, a failure does not
+// claim nothing was written. Here S3 committed it and answered 400.
+func TestPublishRecordSaysAnUncertainWriteMayHaveHappened(t *testing.T) {
+ b := publishFixture(t)
+ box := status.Label(publishMachineID)
+ b.commitThenFail = []int{http.StatusBadRequest}
+ _, err := runPublish(t)
+ if err == nil || !strings.HasPrefix(err.Error(), "the identity record for "+box+" may or may not have been written: ") ||
+ !strings.HasSuffix(err.Error(), ". it is safe to run again, which only ever creates the record and says what is there") {
+ t.Fatalf("err = %v", err)
+ }
+ if _, ok := b.model.Current(publishBucket, boxid.Key(box)); !ok {
+ t.Fatal("the fake did not commit the create")
+ }
+ // The second run finds this box's evidence and says so.
+ if line, err := runPublish(t); err != nil || !strings.Contains(line, "is already in place") {
+ t.Fatalf("second run: line=%q err=%v", line, err)
+ }
+}
+
+// A failure before any create reached the bucket says nothing was written.
func TestPublishRecordReportsAReadFailureAsRetryable(t *testing.T) {
b := publishFixture(t)
box := status.Label(publishMachineID)
- _, err := publishRecordOnce(context.Background(), os.Getenv, publishMachineID, openDeployStoreEnv,
- func(string) (boxid.Record, error) { return boxid.Record{}, errors.New("IMDS down") })
+ run := publishRun()
+ run.read = func(string) (boxid.Record, error) { return boxid.Record{}, errors.New("IMDS down") }
+ _, err := run.publish(context.Background())
want := "the identity record for " + box + " was not published: IMDS down. nothing was written, so it is safe to run again"
if err == nil || err.Error() != want {
t.Fatalf("err = %v\nwant %q", err, want)
@@ -273,48 +417,116 @@ func TestPublishRecordReportsAReadFailureAsRetryable(t *testing.T) {
}
}
-func TestPublishRecordNeedsADeploymentStore(t *testing.T) {
- publishFixture(t)
- t.Setenv("CARLOS_DEPLOYMENT_BUCKET", "")
- _, err := runPublish(t)
- if err == nil || !strings.HasPrefix(err.Error(), "the deployment bucket could not be opened: ") {
- t.Fatalf("err = %v", err)
+// refusedEarly runs the verb with every later seam tripwired: it must
+// refuse before the machine id, the store or IMDS is touched.
+func refusedEarly(t *testing.T, b *boxRecordsBucket, run boxPublishRun, want string) {
+ t.Helper()
+ touched := []string{}
+ run.machineID = func() string { touched = append(touched, "machine id"); return publishMachineID }
+ run.open = func(context.Context, func(string) string) (release.Store, error) {
+ touched = append(touched, "store")
+ return nil, errors.New("opened")
+ }
+ run.read = func(string) (boxid.Record, error) { touched = append(touched, "IMDS"); return boxid.Record{}, nil }
+ _, err := run.publish(context.Background())
+ if err == nil || err.Error() != want {
+ t.Fatalf("err = %v\nwant %q", err, want)
+ }
+ if len(touched) != 0 || len(b.requests()) != 0 {
+ t.Fatalf("read %v and sent %d requests before refusing", touched, len(b.requests()))
}
}
-// recordingStore counts every call that reaches it.
-type recordingStore struct {
- release.Store
- calls []string
+// Census ruling R1: root is refused before any store, IMDS or host read.
+func TestPublishRecordRefusesRoot(t *testing.T) {
+ b := publishFixture(t)
+ run := publishRun()
+ run.geteuid = func() int { return 0 }
+ refusedEarly(t, b, run, "carlos system box publish-record runs as the agent user, not root. run it with systemd-run -p User=carlos.")
}
-func (r *recordingStore) Get(ctx context.Context, key string) ([]byte, string, error) {
- r.calls = append(r.calls, "Get "+key)
- return r.Store.Get(ctx, key)
+func TestPublishRecordRefusesAFleetBox(t *testing.T) {
+ b := publishFixture(t)
+ t.Setenv("CARLOS_FLEET", "acme")
+ t.Setenv("CARLOS_FLEET_BOX", "kass-1")
+ refusedEarly(t, b, publishRun(), boxRecordFleetRefusal)
}
-func (r *recordingStore) Put(ctx context.Context, key string, body []byte, ifMatch string) (string, error) {
- r.calls = append(r.calls, "Put "+key+" "+ifMatch)
- return r.Store.Put(ctx, key, body, ifMatch)
+// Astra P2 (2): a deployment directory cannot refuse an overwrite
+// (DirStore.Put checks, then renames), so only an S3 bucket will do.
+func TestPublishRecordRequiresAnS3Bucket(t *testing.T) {
+ b := publishFixture(t)
+ dir := t.TempDir()
+ t.Setenv("CARLOS_DEPLOYMENT_BUCKET", "")
+ t.Setenv("CARLOS_DEPLOYMENT_DIR", dir)
+ refusedEarly(t, b, publishRun(), boxRecordNoBucket)
+ if entries, _ := os.ReadDir(dir); len(entries) != 0 {
+ t.Fatalf("the deployment directory was written: %v", entries)
+ }
+ // Belt and braces: a store that is not S3 is refused even with a
+ // bucket named.
+ t.Setenv("CARLOS_DEPLOYMENT_BUCKET", publishBucket)
+ run := publishRun()
+ run.open = func(context.Context, func(string) string) (release.Store, error) {
+ return release.NewDirStore(dir), nil
+ }
+ if _, err := run.publish(context.Background()); err == nil || err.Error() != boxRecordNoBucket {
+ t.Fatalf("err = %v", err)
+ }
+ if entries, _ := os.ReadDir(dir); len(entries) != 0 {
+ t.Fatalf("the deployment directory was written: %v", entries)
+ }
}
-func (r *recordingStore) Delete(ctx context.Context, key string) error {
- r.calls = append(r.calls, "Delete "+key)
- return r.Store.Delete(ctx, key)
+// Astra coverage: a box its region's group launched publishes under its
+// group reference, the label boxRecordLoop takes (metricsBoxLabel).
+func TestPublishRecordUsesTheGroupReference(t *testing.T) {
+ b := publishFixture(t)
+ t.Setenv("CARLOS_BOX_REFERENCE", "group")
+ withGroupIdentity(t, "111122223333", "i-0abc", nil)
+ ref := hostactors.GroupBoxReference("111122223333", "i-0abc")
+ if loop := metricsBoxLabel(os.Getenv, publishMachineID); loop != ref {
+ t.Fatalf("the loop's label is %q, want the group reference %q", loop, ref)
+ }
+ line, err := runPublish(t)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if want := "published the identity record for " + ref + " (instance i-0abc) to control/boxes/" + ref + ".json"; line != want {
+ t.Fatalf("line = %q\nwant %q", line, want)
+ }
+ raw, ok := b.model.Current(publishBucket, boxid.Key(ref))
+ var rec boxid.Record
+ if !ok || json.Unmarshal(raw, &rec) != nil || rec.Box != ref {
+ t.Fatalf("record under the group reference: %s", raw)
+ }
+ if _, ok := b.model.Current(publishBucket, boxid.Key(status.Label(publishMachineID))); ok {
+ t.Fatal("a group box published under its machine-id label")
+ }
}
-// The store publish-record hands publishBoxRecord refuses, before the
-// bucket, everything but reading and creating this box's own record.
+// Census ruling R4: the store publish-record hands publishBoxRecord sends
+// the bucket nothing for a delete, a list, an overwrite or any other key.
func TestBoxRecordStoreOnlyReadsAndCreatesItsOwnKey(t *testing.T) {
+ b := publishFixture(t)
ctx := context.Background()
- inner := &recordingStore{Store: release.NewDirStore(t.TempDir())}
+ inner, err := openDeployStoreEnv(ctx, os.Getenv)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, ok := inner.(*release.S3Store); !ok {
+ t.Fatalf("store is %T, want the S3Store", inner)
+ }
s := &boxRecordStore{inner: inner, key: boxid.Key("annie-welding")}
- if _, err := s.Put(ctx, boxid.Key("annie-welding"), []byte("{}"), "etag"); !errors.Is(err, errBoxRecordStore) {
- t.Errorf("replace-only Put: %v", err)
+ if _, err := s.Put(ctx, boxid.Key("annie-welding"), []byte("{}"), `"etag"`); !errors.Is(err, errBoxRecordStore) {
+ t.Errorf("overwrite (replace-only Put): %v", err)
}
if _, err := s.Put(ctx, boxid.Key("bert-sawing"), []byte("{}"), ""); !errors.Is(err, errBoxRecordStore) {
t.Errorf("Put on another box's key: %v", err)
}
+ if _, err := s.Put(ctx, "control/serial", []byte("{}"), ""); !errors.Is(err, errBoxRecordStore) {
+ t.Errorf("Put on another key: %v", err)
+ }
if _, _, err := s.Get(ctx, boxid.Key("bert-sawing")); !errors.Is(err, errBoxRecordStore) {
t.Errorf("Get on another box's key: %v", err)
}
@@ -327,19 +539,22 @@ func TestBoxRecordStoreOnlyReadsAndCreatesItsOwnKey(t *testing.T) {
if _, err := s.ListDirs(ctx, "control/"); !errors.Is(err, errBoxRecordStore) {
t.Errorf("ListDirs: %v", err)
}
- if len(inner.calls) != 0 || s.wrote {
- t.Fatalf("refused calls reached the store: %v (wrote=%v)", inner.calls, s.wrote)
+ if reqs := b.requests(); len(reqs) != 0 || s.attempted || s.wrote {
+ t.Fatalf("refused calls reached S3: %v (attempted=%v wrote=%v)", reqs, s.attempted, s.wrote)
}
if _, err := s.Put(ctx, boxid.Key("annie-welding"), []byte("{}"), ""); err != nil || !s.wrote {
t.Fatalf("create: err=%v wrote=%v", err, s.wrote)
}
if _, err := s.Put(ctx, boxid.Key("annie-welding"), []byte("{}"), ""); !errors.Is(err, release.ErrCASConflict) {
- t.Fatalf("a second create: %v, want the store's ErrCASConflict", err)
+ t.Fatalf("a second create: %v, want the bucket's 412 as ErrCASConflict", err)
+ }
+ if reqs := b.requests(); len(reqs) != 2 || reqs[0].IfNoneMatch != "*" || reqs[1].IfNoneMatch != "*" {
+ t.Fatalf("requests %v, want two create-only PUTs", reqs)
}
}
// boxRecordLoop's gate is unchanged: with CARLOS_REGION_HA off, or on a
-// fleet box, it returns at once and never reaches the bucket. Before the
+// fleet box, it returns at once and never reaches the bucket. Past the
// gate it would retry each minute, so a broken gate shows as a timeout.
func TestBoxRecordLoopGateIsUnchanged(t *testing.T) {
for name, env := range map[string]map[string]string{
diff --git a/internal/provision/box_publish_record_census_test.go b/internal/provision/box_publish_record_census_test.go
index 059346ee2..107a68afe 100644
--- a/internal/provision/box_publish_record_census_test.go
+++ b/internal/provision/box_publish_record_census_test.go
@@ -1,16 +1,18 @@
package provision
// plan14BoxPublishRecordWriteCensusInventory is the record-write inventory
-// with carlos system box publish-record's one write site.
-// PROPOSED-box-publish-record: platform-1e rules on it
-// (docs/superpowers/notes/census-proposal-box-publish-record.md).
+// with carlos system box publish-record's one write site, under
+// census-ruling-box-publish-record (platform-1e, R1-R4;
+// docs/superpowers/notes/census-proposal-box-publish-record.md).
//
// boxRecordStore.Put is the store publish-record hands publishBoxRecord. It
// writes control/boxes/<box>.json, a box's identity record, never an
// instance or sidecar record, and only create-only: a Put on any other key,
-// or with an etag, is refused before the store. Hence nonrecord.
+// or with an etag, is refused before the store. Hence nonrecord, on R4's
+// condition that boxRecordStore is the only store the verb hands
+// publishBoxRecord.
func plan14BoxPublishRecordWriteCensusInventory() []censusEntry {
return append(plan14ActivationToolsWriteCensusInventory(), censusEntry{
- "internal/host/boxpublish.go", "boxRecordStore.Put", "nonrecord", "def:Put,sel:Put", "e5628ec8d3c7e22f88ab877209f578bfbc49f8d7a2d444da42f649625d643ea8",
+ "internal/host/boxpublish.go", "boxRecordStore.Put", "nonrecord", "def:Put,sel:Put", "866978d6c6786a351bbeaf6f9e5382df0b3fd08cbb47aed0c01738f7c5120d7a",
})
}
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go
index 16b35522c..397cfddf4 100644
--- a/internal/provision/plan14_root_reads_test.go
+++ b/internal/provision/plan14_root_reads_test.go
@@ -828,9 +828,10 @@ func TestPlan14RootReadCensus(t *testing.T) {
}
seal = "fc584e113d0c6786bdeb3344354ae567c112c607e68f85056352545557a91b5d"
// Census overlay box-publish-record (carlos system box publish-record),
- // loaded last. PROPOSED-box-publish-record: platform-1e rules on it.
- // Envelope-only: it re-pins the internal/host and root package
- // envelopes, adds and changes no root-read unit, and the seal stands.
+ // loaded last, under census-ruling-box-publish-record (platform-1e,
+ // R1-R4): root unreachable, the verb refuses euid 0 (R1). Envelope-only:
+ // it re-pins the internal/host and root package envelopes, adds and
+ // changes no root-read unit, and the seal stands.
if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-box-publish-record.tsv"), &inv, true); err != nil {
t.Fatal(err)
}
diff --git a/internal/provision/testdata/plan14-root-reads-box-publish-record.tsv b/internal/provision/testdata/plan14-root-reads-box-publish-record.tsv
index 8489d7727..af5c6d515 100644
--- a/internal/provision/testdata/plan14-root-reads-box-publish-record.tsv
+++ b/internal/provision/testdata/plan14-root-reads-box-publish-record.tsv
@@ -1,6 +1,7 @@
-# Plan 14 census overlay box-publish-record (carlos system box publish-record, approved by Paul; owner platform-1e as the host claim's owner), loaded last, after region-ha-2b. PROPOSED-box-publish-record: every row below is proposed, not ruled; platform-1e rules on them (docs/superpowers/notes/census-proposal-box-publish-record.md).
-# internal/host boxpublish.go: RunSystemBoxPublishRecord and publishRecordOnce make one publishBoxRecord attempt, without boxRecordLoop's CARLOS_REGION_HA gate, through boxRecordStore, which admits only Get and create-only Put (If-None-Match: *) on control/boxes/<box>.json and refuses List, ListDirs, Delete, replace-only Put and every other key before the bucket. It refuses on a fleet box (shouldSkipPublisherElection) before any store or IMDS read. Its reads are the existing units it calls, unchanged: metricsBoxLabel and readMachineID for the box's label, openDeployStoreEnv for the deployment store, readIdentityRecord and imdsGetRaw (accepted-legacy) for IMDS, publishBoxRecord and boxid.Load for the one record key. The scan finds no read site in the new units, so no root-read unit is added or changed; boxrecord.go is not edited, and boxRecordLoop's gate is pinned by TestBoxRecordLoopGateIsUnchanged.
+# Plan 14 census overlay box-publish-record (carlos system box publish-record, approved by Paul; owner platform-1e as the host claim's owner), loaded last, after region-ha-2b. RULED by platform-1e as census-ruling-box-publish-record (R1-R4); the ruling and its reasons are docs/superpowers/notes/census-proposal-box-publish-record.md.
+# root unreachable: verb refuses euid 0 (R1). boxPublishRun.publish's first check is the effective uid, before any store, IMDS or host read (the machine id included), so root never reaches publishBoxRecord through this verb. The one supported way to run it is systemd-run --wait --pipe -p EnvironmentFile=/etc/carlos/host.env -p User=carlos /opt/carlos/carlos system box publish-record (R2). It has no region HA gate, and run after the loop has published it finds exactly that evidence and writes nothing (R3). boxRecordStore is the only store it hands publishBoxRecord, and it sends the bucket nothing for a delete, a list, an overwrite or any other key (R4).
+# internal/host boxpublish.go: one publishBoxRecord attempt, without boxRecordLoop's CARLOS_REGION_HA gate. It refuses a fleet box (shouldSkipPublisherElection) and any box without an S3 deployment bucket (CARLOS_DEPLOYMENT_BUCKET unset, or a store that is not release.S3Store: a deployment directory's check-then-rename cannot refuse an overwrite). It reads the record already under control/boxes/<box>.json through boxRecordStore and accepts it only with exactly this box's identity document and signature (sameEvidence; only written_at may differ); with nothing there it calls publishBoxRecord through boxRecordStore, which admits only Get and create-only Put (If-None-Match: *) on that key. Its reads are existing units, unchanged: metricsBoxLabel and readMachineID for the label, openDeployStoreEnv for the store, readIdentityRecord and imdsGetRaw (accepted-legacy) for IMDS, publishBoxRecord and boxid.Load for the record. The scan finds no read site in the new units, so no root-read unit is added or changed. boxrecord.go is not edited, and boxRecordLoop's gate is pinned by TestBoxRecordLoopGateIsUnchanged.
# Root package (system_roles.go, system.go): runSystemBoxVerb hands `box publish-record` to host.RunSystemBoxPublishRecord before roleArgs; systemUsage gains its synopsis and description lines. No root read is added.
-# Envelope-only: the internal/host and root package envelopes are re-pinned; the legacy seal stands. The record-write census (9B8) gains internal/host/boxpublish.go:boxRecordStore.Put, proposed nonrecord (box_publish_record_census_test.go).
-@coverage . c4e6efe2958853ba6480f680da813895cfeb7103fdbeb0977e84bf9fe856a179
-@coverage internal/host 03d139a8f15177642b96900fea16265dd274dd7289b3fd42444c61f4f7ec9327
+# Envelope-only: the internal/host and root package envelopes are re-pinned; the legacy seal stands at fc584e11. The record-write census (9B8) gains internal/host/boxpublish.go:boxRecordStore.Put, nonrecord under R4 (box_publish_record_census_test.go).
+@coverage . d123dce752a02f9db9b87f591d8efab5c37148caff628f596968b2df3b875402
+@coverage internal/host ef673c8ceb936a4164829bf41c65b77c6247eed92b28f1888305313a6bd560ac
diff --git a/system.go b/system.go
index bd1ecb91a..1828c1f5a 100644
--- a/system.go
+++ b/system.go
@@ -167,9 +167,9 @@ The platform's own binary (ops account owners and CARLOS_SYSTEM_OPERATORS):
the same for every box a region's group has enrolled, and for
every box it enrolls later
box publish-record
- run on a box, as root or the agent user: publish its AWS-signed
- identity record to the deployment bucket, once. It never
- replaces or deletes a record
+ run on a box as the agent user, with systemd-run -p User=carlos:
+ publish its AWS-signed identity record to the deployment
+ bucket, once. It never replaces or deletes a record
A box a region's group launches (region HA):
diff --git a/system_roles_test.go b/system_roles_test.go
index d38be7bb0..4ac17417d 100644
--- a/system_roles_test.go
+++ b/system_roles_test.go
@@ -23,7 +23,8 @@ func TestSystemVerbKnowsBoxAndGroup(t *testing.T) {
// carlos system box publish-record reaches the box verb, not the console
// role call: run in a child on a fleet box, it prints the fleet refusal
-// and exits 1 before any store, console or IMDS is touched.
+// (or, as root, the root refusal) and exits 1 before any store, console
+// or IMDS is touched.
func TestSystemBoxPublishRecordDispatch(t *testing.T) {
if os.Getenv("CARLOS_TEST_PUBLISH_RECORD_CHILD") == "1" {
runSystem([]string{"box", "publish-record"})
@@ -41,7 +42,11 @@ func TestSystemBoxPublishRecordDispatch(t *testing.T) {
if !errors.As(err, &exit) || exit.ExitCode() != 1 {
t.Fatalf("exit: %v; stdout %q stderr %q", err, stdout.String(), stderr.String())
}
- if got := strings.TrimSpace(stderr.String()); got != "this is a fleet box. fleet boxes publish no identity record, so nothing was written" {
+ want := "this is a fleet box. fleet boxes publish no identity record, so nothing was written"
+ if os.Geteuid() == 0 { // root is refused first (census ruling R1)
+ want = "carlos system box publish-record runs as the agent user, not root. run it with systemd-run -p User=carlos."
+ }
+ if got := strings.TrimSpace(stderr.String()); got != want {
t.Fatalf("stderr = %q", got)
}
}