carlos / platform Public

store: grant on Linode, Backblaze and Hetzner, not only AWS

`carlos store` could only ever grant against AWS. CARLOS_STORE_BACKEND
fatalled on any value but "aws", and the one Provisioner was S3 + IAM —
so on Hetzner, Linode or Backblaze the bucket half would have worked and
the key half had nothing to call. (The control plane itself has been
provider-agnostic since internal/s3compat; only the per-app store was
stuck.)

The seam split. Creating, emptying and deleting a bucket is plain S3 and
identical everywhere, so it moves to one shared s3Buckets every backend
embeds. Only the CREDENTIAL half varies, which is now the only half a new
backend writes. Two AWS spellings that had leaked into the walk go with
it: the "arn:aws:s3:::" prefix and the IAM user path are the AWS
backend's business, and the seam passes a Target — principal plus bucket
— because different providers key on different halves.

Four backends:

  aws     unchanged: an IAM user per app, confined by an inline policy.
  linode  a key per app, confined by bucket_access at creation.
  b2      an application key per app, confined to one bucket id.
  s3      any S3-compatible store with NO key API.

That last one is Hetzner, and it needed a different arrangement rather
than a fourth copy of the same one: Hetzner's S3 keys exist only in its
web console, and are valid for every bucket in the project. CARLOS
cannot mint one, so it says so — Capabilities().MintsKeys — and grant
takes the pair the operator made by hand.

Because CARLOS did not issue that key it cannot confine it by issuing it
narrowly, so confinement comes from the other direction: every bucket
CARLOS provisions denies every principal but its own app's key and the
console's. App A's key cannot reach app B's bucket because B's bucket
says so.

A policy nobody checks is a promise nobody keeps, so AdoptKey proves it.
It writes the allow-list WITHOUT the new key, requires the store to deny
that key, then adds it and requires the store to allow it. A provider
whose principals are spelled differently (Hetzner scopes them by
project) fails the first probe and the grant refuses rather than
delivering a project-wide key while status claims per-bucket isolation.
CARLOS_STORE_PRINCIPAL_TEMPLATE is how an operator says the spelling;
CARLOS_STORE_BUCKET_POLICY=off is the deliberate escape hatch, and it
downgrades the reported isolation to "none" rather than going quiet.

The honesty rules the CLI already followed now extend to what a backend
can't do. `store status` reports mints_keys and isolation, so a member
learns that rotation here needs an operator BEFORE the 2am they need it,
and `rotate --finish` says "revoked" rather than "deleted" where the key
survives in the operator's account — a deleted-that-wasn't is a live
credential nobody goes back to clean up.

Two ordering bugs found while wiring it, both in Grant. The credential
gate has to sit after the idempotence short-circuit, or re-running grant
over a healthy store — the documented repair for drift — would demand a
key pair for a no-op. And the grant block has to be written after every
refusal that costs nothing, or a grant refused for want of a credential
leaves the mark release.PurgeApp reads as "this app still has backend
estate". It stays before EnsureBucket either way: §5's rule is that
nothing exists in a cloud account which no record names.

Also: EnsureBucket stops sending an AWS LocationConstraint to stores that
select their region by hostname, and settles BucketAlreadyExists with a
HeadBucket, without which a re-grant is not re-runnable on a store that
reports the plainer code. CARLOS_STORE_ACCESS_KEY/SECRET_KEY give the
console an explicit store identity, since the ambient AWS chain is the
wrong one as soon as the store and the control-plane bucket are on
different providers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dTTB7rmQ1q9Dr4ZHb77cn
James Adam pushed by james@keymail.dev 26420341d9b52c3fec0c6eb811975715e68f3476 parent ceecb3a
26 files changed, +4368 −407
  • apimode.go +35 −12
  • cmd/console/main.go +121 −18
  • docs/site/cli.md +45 −2
  • internal/console/apistore.go +115 −10
  • internal/console/apistore_test.go +137 −2
  • internal/console/deletion_test.go +4 −4
  • internal/console/economicspass.go +13 −6
  • internal/console/server.go +10 −0
  • internal/objstore/aws.go +65 −195
  • internal/objstore/aws_test.go +31 −23
  • internal/objstore/b2.go +491 −0
  • internal/objstore/b2_test.go +315 −0
  • internal/objstore/linode.go +312 −0
  • internal/objstore/linode_test.go +244 −0
  • internal/objstore/memprovisioner.go +135 −6
  • internal/objstore/memprovisioner_test.go +9 −9
  • internal/objstore/objstore.go +15 −8
  • internal/objstore/provisioner.go +176 −14
  • internal/objstore/s3bucket.go +259 −0
  • internal/objstore/s3bucket_test.go +138 −0
  • internal/objstore/supplied.go +546 −0
  • internal/objstore/supplied_test.go +410 −0
  • internal/objstore/walk.go +169 −30
  • internal/objstore/walk_test.go +316 −44
  • store.go +101 −15
  • store_test.go +156 −9

CI failed — run details

apimode.go

Loading diff…

cmd/console/main.go

Loading diff…

docs/site/cli.md

Loading diff…

internal/console/apistore.go

Loading diff…

internal/console/apistore_test.go

Loading diff…

internal/console/deletion_test.go

Loading diff…

internal/console/economicspass.go

Loading diff…

internal/console/server.go

Loading diff…

internal/objstore/aws.go

Loading diff…

internal/objstore/aws_test.go

Loading diff…

internal/objstore/b2.go

Loading diff…

internal/objstore/b2_test.go

Loading diff…

internal/objstore/linode.go

Loading diff…

internal/objstore/linode_test.go

Loading diff…

internal/objstore/memprovisioner.go

Loading diff…

internal/objstore/memprovisioner_test.go

Loading diff…

internal/objstore/objstore.go

Loading diff…

internal/objstore/provisioner.go

Loading diff…

internal/objstore/s3bucket.go

Loading diff…

internal/objstore/s3bucket_test.go

Loading diff…

internal/objstore/supplied.go

Loading diff…

internal/objstore/supplied_test.go

Loading diff…

internal/objstore/walk.go

Loading diff…

internal/objstore/walk_test.go

Loading diff…

store.go

Loading diff…

store_test.go

Loading diff…