amadan

carlos / platform Public

Merge the AWS principal report (feat/carloku-principal-inventory-20261002 at 4a1a92d9): reviewed seven rounds, accepted by both reviewers at round seven; a report for a person, unwired

internal/objstore/principals.go and its tests: PrincipalReport.BucketPrincipals,
one listing of the account's authorization details evaluated against one
bucket name, a report of who reaches it and how; not a GranteeProber, so the
grant walk on AWS claims nothing, as before. The design note names what a
completeness claim needs instead (decision 14; proposals A and B), and the
attestation plan carries a superseded-in-part banner. Gate passed alone on
4a1a92d9 (00:36Z) and CI passed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude c4ee880714fbd4736271df566a3d1dcfe20ae089 parent 556278aparent 4a1a92d
7 files changed, +1942 −7
  • docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md +3 −0
  • docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md +1 −1
  • docs/superpowers/plans/2026-10-02-aws-principal-inventory.md +54 −0
  • internal/objstore/principals.go +1132 −0
  • internal/objstore/principals_export_test.go +9 −0
  • internal/objstore/principals_test.go +732 −0
  • internal/objstore/provisioner.go +11 −6

CI cancelled — superseded by a newer push — run details

diff --git a/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md b/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md
index e64f0b429..2cb905898 100644
--- a/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md
+++ b/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md
@@ -1,5 +1,8 @@
# Attesting the ownership history of existing and adopted store buckets
+> **Superseded in part, 2026-10-03.** "What can be proved" §2 and "The mechanism" step 2 below say that reading IAM now gives the complete set of principals that can reach a bucket, and that a ledger can claim completeness from the instant of that read. That is not so: a listing of the account is paged and eventually consistent, a role's policy is not everyone who can act as it, and an absent bucket is not one this grant creates (`2026-10-02-aws-principal-inventory.md`, which supersedes those two passages). The principal report on that branch supports a person's investigation and cannot vouch for an attestation. What a completeness claim needs instead is set out there, as proposals for the owner (decision 14).
+
+
Date: 2026-10-01. Status: proposal with the evidence it would rest on; not built. It adds a billing authority path, so it follows the review of everything already on the branch.
## The problem
diff --git a/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md b/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md
index 86084da53..0167676d4 100644
--- a/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md
+++ b/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md
@@ -45,7 +45,7 @@ Verification after apply: within 24 hours `billing/cur/carloku-transfer/metadata
- The raw result is `logs/aws-ce-s3-2026-07..09.json` in the run directory. It was one Cost Explorer request, which AWS bills at one cent.
- **Credits and discounts hold the month.** AWS writes `Credit`, `Refund`, `Fee`, `EdpDiscount`, `PrivateRateDiscount` and `BundledDiscount` lines on the same transfer usage types as the usage they adjust. Any such line with a non-zero cost holds store transfer for the month for every account (`transfer_adjustment`), because AWS then charged something other than what the usage lines say, and nothing lets an operator release that hold. None appeared from July to September 2026 (above). Check again when the export is approved, and at the payer account, where an organization's credits are applied: if one ever touches S3 transfer the meter will not complete as built, and the classifier needs a ruling on how a discounted byte is charged on.
- **A store's first clock hour never resolves.** A ledger answers for hours that begin at or after the instant it began vouching. A charged line in the hour a store was minted in has no provable owner and holds the month, like any other unowned line. With no free allowance in this account, a store that serves a download in its first hour does that. Nothing releases such a hold today. The choices are an operator release door for a named bucket-hour, or a ruling that the first partial hour of a new store is the platform's cost. That is a product decision; until it is made the code holds.
-- **Existing stores on AWS never complete yet.** A ledger claims completeness only when the provisioner can list every principal whose policy names the bucket and finds none. The AWS provisioner cannot list principals today (the console role has no IAM list permission), so on AWS no ledger claims completeness, new stores included, and store transfer holds. The principal inventory is the first step of the ownership attestation plan (`2026-10-01-bucket-ownership-attestation.md`).
+- **Existing stores on AWS never complete yet.** A ledger claims completeness only when the provisioner can list every principal whose policy names the bucket and finds none. On AWS no listing can show that (`2026-10-02-aws-principal-inventory.md`: a listing is not an instant, a policy is not everyone who can act as its principal, an absent bucket is not one this grant creates), so no ledger on AWS claims completeness, new stores included, and store transfer holds until the account enforces a rule and the bucket's creation is provably the grant's own (proposals A and B there). The principal report built on that branch supports a person's investigation; it is not the mechanism.
- The statements catch-up revisits every month in a 36-month window that is open, was never written, or is missing an account's statement. It costs one read per account per metered month in a console's first pass and nothing after a statement has been seen closed.
- The per-month rebuild door for a statement closed before this existed is still missing (requirements audit step 1).
diff --git a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md
new file mode 100644
index 000000000..97cf5362c
--- /dev/null
+++ b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md
@@ -0,0 +1,54 @@
+# The AWS principal report, and what a completeness claim needs
+
+Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s) its fourth (Opus: accept with fixes, two P2s required; Astra: reject, three P2s), its fifth (both accept with fixes: one P2 each on the possible-reach rule's text reading, and the host role's reach in decision 14) and its sixth (Opus: accept; Astra: accept with fixes, one P2: Object Lambda's management actions, which are in the s3 namespace, on an Object Lambda resource). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS.
+
+## Why this exists
+
+Object-store transfer is billed to a tenant only if the bucket's ownership ledger can claim it names everyone who could ever reach the bucket (`objstore.GranteeProber`, `OwnerAt`). On AWS nothing can make that claim today, so charged store transfer holds the month (`OUTSTANDING.md` item 2).
+
+The first build of this branch tried to make the claim by reading the account: list every principal, evaluate every policy, and call an empty answer "nobody else". The review showed that an empty answer from one listing is not that claim, for three reasons that no amount of care in the reading removes:
+
+1. **A listing is not an instant.** IAM is eventually consistent: a policy attached before the listing began can be missing from it. The listing is paged: a principal on page one can be given access before the last page is read. Nothing in the answer says it is complete.
+2. **A principal's policy is not everyone who can act as it.** A role can be assumed by whoever its trust policy names, another account included; by whoever may pass it to a service or run code where it already runs; and whoever can change IAM can grant themselves the bucket. Setting a role aside as "the platform's" sets aside everyone who can become it.
+3. **An absent bucket is not a bucket this grant creates.** Another actor in the account can create the name, with a policy of its own, between any probe and the walk's own create; AWS answers the walk's create as a success.
+
+So the branch now does less, and says so.
+
+## What is built
+
+`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, a Storage Lens configuration, an Object Lambda access point, a type it does not know; marked so, never dropped; nothing S3 names is excluded, since a Storage Lens configuration exports into a bucket of its owner's choosing), through which policies, a group's possible reach carried to its members as possible, and for each role, the principal values its trust policy's Allow statements name, with a legend for what each spelling means (not who can in fact assume it, which the report does not determine). Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. A resource pattern is read two ways, as text (a star or a question mark free to run across colons) and as the six fields of an ARN with the ones an incomplete pattern leaves out as stars, and a reach under either is reported; the possible-reach rule is one text test on both forms (can the pattern spell `arn:<partition>:s3:` or `arn:<partition>:s3::` and then a character other than a colon, or anything under `arn:<partition>:s3-object-lambda:`); which reading IAM gives an incomplete pattern is unverified. The legend lives in the Go type's comment; the JSON carries none, and whatever prints the report for a person must print it.
+
+- It is **not** a `GranteeProber`, and a test says so. The grant walk on AWS claims nothing, as before. It is a report to support a person's investigation; it cannot verify that any rule is in force in the account, and identical reports can come from an account with and without one.
+- Within the documents it is given it leans toward listing: a Deny, a condition or a boundary is not evaluated; `NotAction`, `NotResource` and policy variables are taken to reach.
+- What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a null `Id` or `Sid`, a condition that is not a map of at least one operator each mapping at least one key to a scalar or a non-empty list of scalars, a `Principal` in an identity policy, an unknown element, an unknown principal kind in a trust policy); the same principal, managed policy, inline policy name or policy version id returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing.
+- It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs, or an Access Grants location role, may do under their roles, beyond reporting the principal that may start them as a possible reach; who inside the account may assume a role (`sts:AssumeRole` in their own policies), pass it to a service (`iam:PassRole`) or run code where it already runs (a host under SSM or Instance Connect, a function's code, a container: everyone who controls compute running as a listed role reaches what it reaches); who may change IAM (`iam:PutUserPolicy`, `iam:AttachRolePolicy`, `iam:CreatePolicyVersion` and the like) and so grant themselves the bucket in one call; credentials vended by other services (Lake Formation, Identity Center grantees of Access Grants that are not IAM principals); anything in another account; organization policies and permissions boundaries (which only narrow).
+
+Use: a person investigating who reaches a store name today, in support of the decisions below. It is not evidence for an attestation: the attestation plan's claim that an IAM read proves completeness is superseded (a banner at its head says so). There is no harness to run it yet: an operator would need a small Go program in this module that loads their own profile, builds an IAM client, calls `NewPrincipalReport` and `BucketPrincipals` with a deadline, and holds `iam:GetAccountAuthorizationDetails`; from a sandboxed session that also needs network and credential access the sandbox withholds. Nothing in the console calls it.
+
+Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. Both matchers read a resource pattern as text and, when it is an incomplete ARN, as padded fields too, and report a reach under either; which reading IAM gives an incomplete pattern, and whether its wildcards run across colons, is on the list to verify against a real account. The text reading over-reports, and in one common case relabels: any star before a literal an object key can hold runs into one of this bucket's keys, so a grant on `arn:aws:s3:*:123456789012:accesspoint/*` is listed as direct rather than possible, and an incomplete ARN of another service with a star (`arn:*:function:thumbs`) is listed as direct; a question mark cannot run into a key that way and is listed as possible.
+
+Mutation checks, in a tree copy, each a one-line edit of `principals.go` run against the inventory tests, logged in the run directory's logs as `inventory-mutants-round{1..6}.txt`. Rounds one to three record each mutation and the failing tests' names (forty after the first review, one equivalent; twelve after the second; the third's K, L and M entries, with replacements for the ones that first failed to compile); round four records ten mutations and only whether the tests failed. Round five records thirteen mutations against the changed code, each with its exact edit and the failing tests' first assertions: six on the possible-reach text test (the S3 test, the account form, the region form, the non-colon requirement, the padded reading and the Object Lambda test each dropped), the two readings dropped from the direct matcher, the three empty condition shapes accepted, a mixed group propagated as possible, and a nil page dereferenced; round four's Storage Lens exclusion, operator-with-no-key and empty-condition mutations were not rerun as such (the last two are among the thirteen in their round-five form; the exclusion has no counterpart in a rule that reads text). Round six records those thirteen again, the Storage Lens exclusion put back, and S3 actions kept out of the Object Lambda prefix. Every recorded mutation dies; the counts are as logged and nothing more is claimed of them.
+
+## What a completeness claim needs instead (not built; needs the owner)
+
+Looking cannot establish it. A rule the account enforces can. Two parts, each a design question, each stated here with what its review found missing in it:
+
+**A. Nobody else can reach a store bucket.** (Today the host role can give itself any store bucket's objects through a store user it makes: see decision 14 below. The binding this proposal asks for is exactly what that lacks.) Proposed direction: an organization policy (or, failing that, a permissions boundary required on every principal the platform creates plus an account rule) that denies S3 actions on the store bucket prefix to every principal except each bucket's own store user and a short list named by ARN. What that needs, beyond the sentence: a binding between each store identity and its one bucket, enforced (a store user under the path with a policy for another store's bucket must be impossible, not merely absent); coverage of the paths an identity policy does not show (resource policies and grants to other accounts, service-linked roles, access points); the enforcement configuration itself protected from the principals it constrains; and a way to verify the control is in force that is separate from this report, which cannot see it. An organization policy binds neither the organization's management account nor service-linked roles, and a resource control policy, the nearer tool for "nobody but X reaches these buckets", has the same two limits (it does not apply to resources in the management account and does not restrict service-linked roles); so if the flagship account is the management account neither can carry A, and which account the flagship is sits on the list below. Who is on the named list, and who can assume each role on it, is decision 14.
+
+**B. The bucket is the one this grant made.** Proposed direction: the grant mints a bucket name with a random part recorded in its grant block before any provider call, and refuses a bucket that already exists under that name. What that needs, beyond the sentence: randomness only makes an accidental collision and a guess unlikely; an administrator who can read the grant block, or the store user's policy written before the bucket, can create the name first, so creation provenance needs an enforceable restriction on who may create buckets under the prefix, not unpredictability; an explicit treatment of the provider answering an already-owned bucket's create as success (the us-east-1 case the provisioner already notes) and of a retry after the platform's own create; and crash recovery that tells the platform's own earlier create from someone else's.
+
+Neither is designed, and both change infrastructure or the grant walk, which need their own review. Until both, no bucket's ledger on AWS claims completeness and store transfer holds.
+
+## Decision for Paul (item 14, reworded)
+
+Which principals may reach tenant store buckets besides the tenant's own store user, and who can act as each. Today the host role (it creates and deletes store buckets and holds no `s3:GetObject` of its own, but `infra/modules/region-host/main.tf` lets it create a store user and write that user's policy under the store boundary, which allows `s3:GetObject` on `carlos-store-*/*`, and mint the user's keys: so it, and whoever runs code on a host as it, can give itself any store bucket's objects through a user it makes, in three calls) and any administrator role reach every store bucket. AWS charges transfer by bucket and does not say who moved the bytes: if staff download from a tenant's bucket, that transfer lands on the tenant's bill.
+
+Proposed: only the deployment's host role and roles that need account administration to assume; named one by one, by ARN, with each role's trust policy part of the record; staff do not read tenant buckets, and if support ever requires it the platform credits the bytes. And approve the direction in A and B above, or say what should stand in their place.
+
+## Before anything here is wired
+
+1. The owner's answers to item 14 and to A and B.
+2. A real listing, read by a person: encodings, service-linked roles, Identity Center roles, AWS managed policies with constructs this parser refuses. The first real run may error.
+3. The resource side for existing buckets (bucket policy, ACL, access points) and the attestation door.
+4. IAM's own matching, which this parser guesses at: whether an incomplete resource ARN is padded, read as text or refused when the policy is written; whether `*` runs across colons; whether the service field of a resource ARN is case-folded; and whether real documents carry shapes this parser refuses (an empty condition, an operator with no key, an empty list of values), which would make the first real run an error rather than a report.
+5. Whether the flagship account is its organization's management account, which decides whether proposal A can stand on an organization policy.
diff --git a/internal/objstore/principals.go b/internal/objstore/principals.go
new file mode 100644
index 000000000..6e7bcb99f
--- /dev/null
+++ b/internal/objstore/principals.go
@@ -0,0 +1,1132 @@
+package objstore
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "net/url"
+ "sort"
+ "strings"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ "github.com/aws/aws-sdk-go-v2/service/iam"
+ iamtypes "github.com/aws/aws-sdk-go-v2/service/iam/types"
+)
+
+// The AWS principal inventory: which users, roles and groups of the
+// account carry an identity policy that reaches a bucket name.
+//
+// It is a report for a person, and it is NOT a GranteeProber
+// (provisioner.go): it claims nothing the ownership ledger could rest on.
+// Its first review said why, and was right:
+//
+// - A listing is not an instant. IAM is eventually consistent, so a
+// policy attached before the listing began can be missing from it;
+// and a principal returned on the first page can be given access
+// before the last page is read. Nothing in the answer says it is
+// complete, and reading it twice does not make it so.
+// - A principal's policy is not everyone who can act as it. A role can
+// be assumed by whoever its trust policy names, another account's
+// principals included; by whoever may pass it to a service; and by
+// whoever can run code where it already runs (a host under SSM, a
+// function's code, a container). Whoever can write IAM can grant
+// themselves the bucket in one call. The trust policy's principal
+// values are reported for each role that reaches the name; none of
+// the rest is traced.
+// - An absent bucket now is not a bucket this grant creates. Another
+// actor in the account can create the name, with a policy of its own,
+// between any probe and the walk's own create, and AWS answers the
+// walk's create as a success.
+//
+// So a claim that nobody else can reach a store bucket cannot be read off
+// the account. It needs a basis the account enforces (a rule that stops
+// such a grant existing, not a look that failed to find one) and a bucket
+// only its own grant could have created. Neither exists; both are design
+// questions put to the owner
+// (docs/superpowers/plans/2026-10-02-aws-principal-inventory.md). Until
+// then no ledger on AWS claims completeness and store transfer holds.
+//
+// What the report does. It reads the whole account's authorization
+// details (iam:GetAccountAuthorizationDetails, every page) and evaluates
+// every identity policy itself: users, roles and groups; inline and
+// attached managed policies (the default version); a user's groups.
+// Within the documents it is given it leans one way, toward listing:
+//
+// - Deny statements, conditions, permissions boundaries, session
+// policies and organization policies only narrow access. None is
+// evaluated. A principal an explicit Deny keeps out is still listed.
+// - NotAction and NotResource on an Allow are taken to reach the bucket.
+// - A policy variable (${aws:username}), and its escapes, are taken to
+// expand to anything.
+// - A group whose policy reaches the bucket is listed, with or without
+// members, and so is each of its members.
+// - A grant on any S3 resource that is not a general-purpose bucket or
+// an object in one (an access point, a multi-region access point, an
+// Object Lambda access point, an Access Grants instance, a job, a
+// Storage Lens configuration, a type it does not know) names no
+// bucket in its ARN and can reach a bucket's objects through a
+// relation the ARN does not show: its principal is listed as possibly
+// reaching, marked so, never dropped.
+// - A resource pattern is read two ways, as text and as the six fields
+// of an ARN with the ones an incomplete pattern leaves out as stars;
+// a reach under either is reported. Which reading IAM gives an
+// incomplete pattern is unverified. Under the text reading a star or
+// a question mark is free to run across colons, so a star before a
+// literal an object key can hold runs into one of this bucket's keys
+// and the grant is listed as direct, not possible: a grant on
+// "arn:aws:s3:*:123456789012:accesspoint/*" is listed as direct.
+//
+// What it cannot read is an error, never a shorter list: a document it
+// cannot decode; a JSON member given twice anywhere in a document; a
+// statement or an element that is not the grammar it knows, in an
+// identity policy or a trust policy, Deny statements included (a
+// condition with no operator, an operator with no key and a key with an
+// empty list of values among them); the same principal, policy or inline
+// policy name returned twice; a managed
+// policy whose own default-version id and the version marked default
+// disagree; an attached managed policy the listing did not return; a
+// listing truncated without a marker, or that does not end.
+//
+// What it does not read at all: the root user; a bucket's own policy, ACL
+// and its access points' policies; what Batch Operations and replication
+// jobs, or an Access Grants location role, may do under their roles; who
+// may assume a role from inside the account, pass it to a service, or
+// control compute that runs as it; who may change IAM, and so grant
+// themselves the bucket; credentials other services vend (Lake Formation,
+// Access Grants to grantees that are not IAM principals); anything in
+// another account.
+//
+// Named by the deployment. A deployment may name principals as its own,
+// by exact ARN; the report sets them apart under that name. It is the
+// deployment's word, a label for the reader; it exempts nothing from
+// anything, and a role so named may be assumable by anyone its trust
+// policy lets in.
+
+// IAMAuthorizationLister is the one IAM read the inventory makes.
+// *iam.Client implements it.
+type IAMAuthorizationLister interface {
+ GetAccountAuthorizationDetails(ctx context.Context, in *iam.GetAccountAuthorizationDetailsInput, optFns ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error)
+}
+
+// inventoryMaxPages bounds the listing: a listing that has not ended by
+// then is not trusted to end. At the service's page size of a hundred
+// entities this is far more than an account can hold.
+var inventoryMaxPages = 2000
+
+// PrincipalGrant is one principal whose identity policies reach a bucket
+// name, and the policies through which they do.
+type PrincipalGrant struct {
+ ARN string `json:"arn"`
+ Kind string `json:"kind"` // user | role | group
+ // Via names each policy that reaches the bucket: "inline:<name>",
+ // "managed:<policy arn>", and for a user "group:<group arn>" for each
+ // of its groups that does. Sorted.
+ Via []string `json:"via"`
+ // AssumableBy, for a role, is every principal value its trust
+ // policy's Allow statements name, sorted, whatever their actions and
+ // conditions: not who can in fact assume the role, which the report
+ // does not determine. Read each spelling as:
+ // "AWS:arn:...:root" any principal in that account that may
+ // sts:AssumeRole, not the root user alone;
+ // "AWS:<arn>" that principal; in this account the trust
+ // policy alone is enough, from another account
+ // its own policy must allow it too;
+ // "Service:<name>" that service, and so whoever may iam:PassRole
+ // this role to it and whoever can run code
+ // where it already runs (a host under SSM, a
+ // function's code, a container);
+ // "AWS:<account id>" the same as the root spelling of that account;
+ // "AWS:AROA…", "AWS:AIDA…" a principal IAM deleted after the trust
+ // policy named it (its unique id stays; a
+ // principal recreated under the same name does
+ // not match it);
+ // "Federated:<arn>" anyone that identity provider vouches for;
+ // "Federated:<name>" a web identity provider named by its domain
+ // (cognito-identity.amazonaws.com,
+ // accounts.google.com, ...), the same;
+ // "CanonicalUser:<id>" an S3 canonical user id, which a trust policy
+ // is not expected to carry (reported as read if
+ // one does);
+ // "*" anyone the role's conditions, if any, let in.
+ AssumableBy []string `json:"assumable_by,omitempty"`
+}
+
+// PrincipalInventory is everything one listing showed about one name. It
+// is an observation, not a claim of completeness, and nothing in it is
+// verified against anything but the listing.
+type PrincipalInventory struct {
+ Bucket string `json:"bucket"`
+ // Reaching are the principals whose identity policies reach the name
+ // and that the deployment did not name as its own.
+ Reaching []PrincipalGrant `json:"reaching"`
+ // NamedByDeployment are the principals that reach the name and that
+ // the deployment's configuration names as its own. The name is the
+ // deployment's word, unverified: a role here may be assumable by
+ // anyone its AssumableBy says.
+ NamedByDeployment []PrincipalGrant `json:"named_by_deployment"`
+ // What was read, so an empty answer can be told from an empty account.
+ Users int `json:"users"`
+ Roles int `json:"roles"`
+ Groups int `json:"groups"`
+ ManagedPolicies int `json:"managed_policies"`
+}
+
+// PrincipalReport reads the account and reports on bucket names.
+type PrincipalReport struct {
+ api IAMAuthorizationLister
+ platform map[string]bool
+}
+
+// NewPrincipalReport builds the report over an IAM client. platform is
+// the principals the deployment names as its own, each by its exact ARN.
+// Nothing in the console calls this.
+func NewPrincipalReport(api IAMAuthorizationLister, platform []string) (*PrincipalReport, error) {
+ if api == nil {
+ return nil, errors.New("objstore: no IAM client")
+ }
+ named := map[string]bool{}
+ for _, arn := range platform {
+ if err := validPlatformPrincipal(arn); err != nil {
+ return nil, err
+ }
+ named[arn] = true
+ }
+ return &PrincipalReport{api: api, platform: named}, nil
+}
+
+// validPlatformPrincipal accepts the ARN of one user, role or group,
+// spelled out: no wildcard, no pattern. The list is one a person can
+// read, each entry one principal.
+func validPlatformPrincipal(arn string) error {
+ parts := strings.SplitN(arn, ":", 6)
+ if len(parts) != 6 || parts[0] != "arn" || parts[1] == "" || parts[2] != "iam" || parts[3] != "" || len(parts[4]) != 12 || strings.Trim(parts[4], "0123456789") != "" {
+ return fmt.Errorf("objstore: platform principal %q is not an IAM principal ARN", arn)
+ }
+ kind, name, ok := strings.Cut(parts[5], "/")
+ if !ok || name == "" || (kind != "user" && kind != "role" && kind != "group") {
+ return fmt.Errorf("objstore: platform principal %q is not a user, role or group", arn)
+ }
+ if strings.ContainsAny(arn, "*?$ \t\n") {
+ return fmt.Errorf("objstore: platform principal %q is a pattern; each principal is named by its own ARN", arn)
+ }
+ return nil
+}
+
+// BucketPrincipals lists the account once and reports which principals'
+// identity policies reach the name.
+func (p *PrincipalReport) BucketPrincipals(ctx context.Context, name string) (*PrincipalInventory, error) {
+ if name == "" || strings.ContainsAny(name, "*?/:${} ") {
+ return nil, fmt.Errorf("objstore: %q is not a bucket name", name)
+ }
+ account, err := p.listAccount(ctx)
+ if err != nil {
+ return nil, err
+ }
+ return account.reaching(name, p.platform)
+}
+
+// Errors a caller can tell apart: each is a listing that is not trusted,
+// and none is an empty answer.
+var (
+ errInventoryTruncated = errors.New("objstore: the account's authorization details are truncated with no way on")
+ errInventoryEndless = errors.New("objstore: the account's authorization details did not end")
+ errInventoryRepeated = errors.New("objstore: the account's authorization details return the same thing twice")
+)
+
+// accountPolicies is the account's authorization details, decoded.
+type accountPolicies struct {
+ users, roles, groups []inventoryPrincipal
+ managed map[string]*policyDoc // by policy ARN
+}
+
+type inventoryPrincipal struct {
+ arn, kind, name string
+ inline map[string]*policyDoc // by policy name
+ attached []string // managed policy ARNs
+ groups []string // group names, for a user
+ assumableBy []string // a role's trust policy principals
+}
+
+// inlinePolicies decodes a principal's inline policies. The same name
+// twice is refused: the second would replace the first, and one of them
+// might be the one that reaches.
+func inlinePolicies(arn string, list []iamtypes.PolicyDetail) (map[string]*policyDoc, error) {
+ out := map[string]*policyDoc{}
+ for _, d := range list {
+ name := aws.ToString(d.PolicyName)
+ if _, twice := out[name]; twice || name == "" {
+ return nil, fmt.Errorf("%w: inline policy %q of %s", errInventoryRepeated, name, arn)
+ }
+ doc, err := decodePolicy(aws.ToString(d.PolicyDocument))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: policy %q of %s: %w", name, arn, err)
+ }
+ out[name] = doc
+ }
+ return out, nil
+}
+
+func attachedPolicies(list []iamtypes.AttachedPolicy) []string {
+ var out []string
+ for _, a := range list {
+ out = append(out, aws.ToString(a.PolicyArn))
+ }
+ return out
+}
+
+func (p *PrincipalReport) listAccount(ctx context.Context) (*accountPolicies, error) {
+ acct := &accountPolicies{managed: map[string]*policyDoc{}}
+ var marker *string
+ markers := map[string]bool{}
+ principals := map[string]bool{}
+ add := func(list *[]inventoryPrincipal, pr inventoryPrincipal) error {
+ if pr.arn == "" || pr.name == "" {
+ return errors.New("objstore: the account's authorization details name a principal with no ARN or name")
+ }
+ // The same principal twice, on one page or two, is not put
+ // together or chosen between: either copy might be the stale one.
+ if principals[pr.arn] {
+ return fmt.Errorf("%w: %s", errInventoryRepeated, pr.arn)
+ }
+ principals[pr.arn] = true
+ *list = append(*list, pr)
+ return nil
+ }
+ for page := 0; ; page++ {
+ if page >= inventoryMaxPages {
+ return nil, fmt.Errorf("%w within %d pages", errInventoryEndless, inventoryMaxPages)
+ }
+ out, err := p.api.GetAccountAuthorizationDetails(ctx, &iam.GetAccountAuthorizationDetailsInput{Marker: marker})
+ if err != nil {
+ return nil, fmt.Errorf("objstore: list the account's authorization details: %w", err)
+ }
+ if out == nil {
+ return nil, errors.New("objstore: list the account's authorization details: no answer and no error")
+ }
+ for _, u := range out.UserDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(u.Arn), kind: "user", name: aws.ToString(u.UserName), groups: u.GroupList, attached: attachedPolicies(u.AttachedManagedPolicies)}
+ if pr.inline, err = inlinePolicies(pr.arn, u.UserPolicyList); err != nil {
+ return nil, err
+ }
+ if err := add(&acct.users, pr); err != nil {
+ return nil, err
+ }
+ }
+ for _, g := range out.GroupDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(g.Arn), kind: "group", name: aws.ToString(g.GroupName), attached: attachedPolicies(g.AttachedManagedPolicies)}
+ if pr.inline, err = inlinePolicies(pr.arn, g.GroupPolicyList); err != nil {
+ return nil, err
+ }
+ if err := add(&acct.groups, pr); err != nil {
+ return nil, err
+ }
+ }
+ for _, r := range out.RoleDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(r.Arn), kind: "role", name: aws.ToString(r.RoleName), attached: attachedPolicies(r.AttachedManagedPolicies)}
+ if pr.inline, err = inlinePolicies(pr.arn, r.RolePolicyList); err != nil {
+ return nil, err
+ }
+ if pr.assumableBy, err = trustPrincipals(aws.ToString(r.AssumeRolePolicyDocument)); err != nil {
+ return nil, fmt.Errorf("objstore: trust policy of %s: %w", pr.arn, err)
+ }
+ if err := add(&acct.roles, pr); err != nil {
+ return nil, err
+ }
+ }
+ for _, m := range out.Policies {
+ arn := aws.ToString(m.Arn)
+ if _, twice := acct.managed[arn]; twice || arn == "" {
+ return nil, fmt.Errorf("%w: managed policy %q", errInventoryRepeated, arn)
+ }
+ defaults := 0
+ var current *string
+ var currentID string
+ versions := map[string]bool{}
+ for _, v := range m.PolicyVersionList {
+ id := aws.ToString(v.VersionId)
+ if id == "" || versions[id] {
+ return nil, fmt.Errorf("%w: managed policy %s version %q", errInventoryRepeated, arn, id)
+ }
+ versions[id] = true
+ if v.IsDefaultVersion {
+ defaults++
+ current = v.Document
+ currentID = id
+ }
+ }
+ if defaults != 1 || current == nil {
+ return nil, fmt.Errorf("objstore: managed policy %s has %d default versions with a document in the listing; want one", arn, defaults)
+ }
+ // The policy's own record of its default version must name
+ // the version marked default: two parts of one answer that
+ // disagree are not an answer.
+ if want := aws.ToString(m.DefaultVersionId); want == "" || currentID == "" || want != currentID {
+ return nil, fmt.Errorf("objstore: managed policy %s says its default version is %q and the listing marks %q", arn, want, currentID)
+ }
+ doc, err := decodePolicy(aws.ToString(current))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: managed policy %s: %w", arn, err)
+ }
+ acct.managed[arn] = doc
+ }
+ if !out.IsTruncated {
+ break
+ }
+ next := aws.ToString(out.Marker)
+ if next == "" || markers[next] {
+ return nil, errInventoryTruncated
+ }
+ markers[next] = true
+ marker = out.Marker
+ }
+ return acct, nil
+}
+
+// reaching evaluates every principal against one bucket name.
+func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*PrincipalInventory, error) {
+ inv := &PrincipalInventory{Bucket: bucket, Reaching: []PrincipalGrant{}, NamedByDeployment: []PrincipalGrant{}, Users: len(a.users), Roles: len(a.roles), Groups: len(a.groups), ManagedPolicies: len(a.managed)}
+ via := func(pr inventoryPrincipal) ([]string, error) {
+ partition := "aws"
+ if parts := strings.SplitN(pr.arn, ":", 3); len(parts) == 3 && parts[0] == "arn" && parts[1] != "" {
+ partition = parts[1]
+ }
+ var out []string
+ note := func(kind reachKind, what string) {
+ switch kind {
+ case reachDirect:
+ out = append(out, what)
+ case reachPossible:
+ out = append(out, possiblyPrefix+what)
+ }
+ }
+ for name, doc := range pr.inline {
+ note(doc.reaches(partition, bucket), "inline:"+name)
+ }
+ for _, arn := range pr.attached {
+ doc, ok := a.managed[arn]
+ if !ok {
+ return nil, fmt.Errorf("objstore: %s has managed policy %s attached, and the listing did not return it", pr.arn, arn)
+ }
+ note(doc.reaches(partition, bucket), "managed:"+arn)
+ }
+ return out, nil
+ }
+ groupVia := map[string][]string{}
+ groupARN := map[string]string{}
+ add := func(pr inventoryPrincipal, through []string) {
+ if len(through) == 0 {
+ return
+ }
+ sort.Strings(through)
+ g := PrincipalGrant{ARN: pr.arn, Kind: pr.kind, Via: through, AssumableBy: pr.assumableBy}
+ if platform[pr.arn] {
+ inv.NamedByDeployment = append(inv.NamedByDeployment, g)
+ } else {
+ inv.Reaching = append(inv.Reaching, g)
+ }
+ }
+ for _, g := range a.groups {
+ through, err := via(g)
+ if err != nil {
+ return nil, err
+ }
+ if _, twice := groupARN[g.name]; twice {
+ return nil, fmt.Errorf("%w: the group %q", errInventoryRepeated, g.name)
+ }
+ groupVia[g.name], groupARN[g.name] = through, g.arn
+ add(g, through)
+ }
+ for _, u := range a.users {
+ through, err := via(u)
+ if err != nil {
+ return nil, err
+ }
+ for _, name := range u.groups {
+ arn, ok := groupARN[name]
+ if !ok {
+ return nil, fmt.Errorf("objstore: %s is in the group %q, and the listing did not return it", u.arn, name)
+ }
+ if len(groupVia[name]) == 0 {
+ continue
+ }
+ // The member reaches as the group does: if every one of the
+ // group's reaching policies is only a possible reach, so is
+ // the membership.
+ direct := false
+ for _, w := range groupVia[name] {
+ if !strings.HasPrefix(w, possiblyPrefix) {
+ direct = true
+ }
+ }
+ if direct {
+ through = append(through, "group:"+arn)
+ } else {
+ through = append(through, possiblyPrefix+"group:"+arn)
+ }
+ }
+ add(u, through)
+ }
+ for _, r := range a.roles {
+ through, err := via(r)
+ if err != nil {
+ return nil, err
+ }
+ add(r, through)
+ }
+ for _, list := range []*[]PrincipalGrant{&inv.Reaching, &inv.NamedByDeployment} {
+ sort.Slice(*list, func(i, j int) bool { return (*list)[i].ARN < (*list)[j].ARN })
+ }
+ return inv, nil
+}
+
+// policyDoc is one identity policy, as much of it as the inventory reads.
+type policyDoc struct {
+ statements []policyStmt
+}
+
+type policyStmt struct {
+ allow bool
+ actions []string
+ resources []string
+ hasNotAction, hasNotResource bool
+}
+
+// stringOrList decodes an IAM element that is a string or a list of
+// strings, and refuses anything else: null, a number, an empty list, an
+// empty string, a list holding something that is not a string.
+func stringOrList(raw json.RawMessage) ([]string, error) {
+ var one *string
+ if err := json.Unmarshal(raw, &one); err == nil {
+ if one == nil || *one == "" {
+ return nil, errors.New("null or empty where a value is required")
+ }
+ return []string{*one}, nil
+ }
+ var many []*string
+ if err := json.Unmarshal(raw, &many); err != nil {
+ return nil, errors.New("neither a string nor a list of strings")
+ }
+ if len(many) == 0 {
+ return nil, errors.New("an empty list where a value is required")
+ }
+ out := make([]string, 0, len(many))
+ for _, v := range many {
+ if v == nil || *v == "" {
+ return nil, errors.New("null or empty in a list of values")
+ }
+ out = append(out, *v)
+ }
+ return out, nil
+}
+
+// noRepeatedMembers refuses a JSON text in which any object, at any
+// depth, gives one member name twice: the standard decoder keeps the last,
+// and the first may be the one that reaches.
+func noRepeatedMembers(text string) error {
+ dec := json.NewDecoder(strings.NewReader(text))
+ type frame struct {
+ object bool
+ seen map[string]bool
+ expectKey bool
+ }
+ var stack []*frame
+ for {
+ tok, err := dec.Token()
+ if err == io.EOF {
+ return nil
+ }
+ if err != nil {
+ return err
+ }
+ if d, isDelim := tok.(json.Delim); isDelim && (d == '}' || d == ']') {
+ stack = stack[:len(stack)-1]
+ if len(stack) > 0 && stack[len(stack)-1].object {
+ stack[len(stack)-1].expectKey = true
+ }
+ continue
+ }
+ if len(stack) > 0 {
+ top := stack[len(stack)-1]
+ if top.object && top.expectKey {
+ name, _ := tok.(string)
+ if top.seen[name] {
+ return fmt.Errorf("the member %q is given twice", name)
+ }
+ top.seen[name] = true
+ top.expectKey = false
+ continue
+ }
+ }
+ // A value: a scalar, or the start of an object or a list.
+ if d, isDelim := tok.(json.Delim); isDelim {
+ if len(stack) > 0 && stack[len(stack)-1].object {
+ stack[len(stack)-1].expectKey = true
+ }
+ stack = append(stack, &frame{object: d == '{', seen: map[string]bool{}, expectKey: d == '{'})
+ continue
+ }
+ if len(stack) > 0 && stack[len(stack)-1].object {
+ stack[len(stack)-1].expectKey = true
+ }
+ }
+}
+
+// policyText turns a policy document as IAM returns it (percent-encoded
+// JSON, RFC 3986: a plus sign is a plus sign; or plain JSON) into its
+// top-level elements, and checks them against the ones a policy has.
+func policyText(raw string) (map[string]json.RawMessage, error) {
+ text := strings.TrimSpace(raw)
+ if text == "" {
+ return nil, errors.New("an empty policy document")
+ }
+ if !strings.HasPrefix(text, "{") {
+ decoded, err := url.PathUnescape(text)
+ if err != nil {
+ return nil, fmt.Errorf("undecodable policy document: %w", err)
+ }
+ text = decoded
+ }
+ if err := noRepeatedMembers(text); err != nil {
+ return nil, fmt.Errorf("policy document: %w", err)
+ }
+ var doc map[string]json.RawMessage
+ if err := json.Unmarshal([]byte(text), &doc); err != nil || doc == nil {
+ return nil, errors.New("policy document is not a JSON object")
+ }
+ for key := range doc {
+ switch key {
+ case "Version", "Id", "Statement":
+ default:
+ return nil, fmt.Errorf("policy document has the element %q", key)
+ }
+ }
+ if v, ok := doc["Version"]; ok {
+ var version string
+ if err := json.Unmarshal(v, &version); err != nil || (version != "2012-10-17" && version != "2008-10-17") {
+ return nil, fmt.Errorf("policy document has the version %s", string(v))
+ }
+ }
+ if v, ok := doc["Id"]; ok {
+ var id *string
+ if err := json.Unmarshal(v, &id); err != nil || id == nil {
+ return nil, fmt.Errorf("policy document has the id %s", string(v))
+ }
+ }
+ return doc, nil
+}
+
+// statementShape checks the elements of one statement, of an identity
+// policy or a trust policy, that are not evaluated: their shapes must be
+// the grammar's even though their meaning is left alone. A statement is
+// read in full whatever its effect.
+func statementShape(i int, st map[string]json.RawMessage, allowed map[string]bool) error {
+ for key, raw := range st {
+ if !allowed[key] {
+ if key == "Principal" || key == "NotPrincipal" {
+ return fmt.Errorf("statement %d has %s, which an identity policy does not", i, key)
+ }
+ return fmt.Errorf("statement %d has the element %q", i, key)
+ }
+ switch key {
+ case "Sid":
+ var sid *string
+ if err := json.Unmarshal(raw, &sid); err != nil || sid == nil {
+ return fmt.Errorf("statement %d has the sid %s", i, string(raw))
+ }
+ case "Condition":
+ // At least one operator, each a map of at least one key to a
+ // scalar or a non-empty list of scalars. The meaning is not
+ // evaluated; the shape is.
+ var cond map[string]map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &cond); err != nil || len(cond) == 0 {
+ return fmt.Errorf("statement %d has a condition that is not a map of at least one operator", i)
+ }
+ for op, keys := range cond {
+ if len(keys) == 0 {
+ return fmt.Errorf("statement %d condition %s is null or has no key", i, op)
+ }
+ for k, v := range keys {
+ if !conditionValue(v) {
+ return fmt.Errorf("statement %d condition %s %s has a value that is not a scalar or a list of scalars", i, op, k)
+ }
+ }
+ }
+ }
+ }
+ return nil
+}
+
+// policyStatements gives a document's statements, each as its elements:
+// one statement or a list of at least one, never null.
+func policyStatements(doc map[string]json.RawMessage) ([]map[string]json.RawMessage, error) {
+ raw, ok := doc["Statement"]
+ if !ok {
+ return nil, errors.New("policy document has no Statement")
+ }
+ var list []map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &list); err != nil {
+ var one map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &one); err != nil {
+ return nil, errors.New("policy Statement is neither a statement nor a list of them")
+ }
+ list = []map[string]json.RawMessage{one}
+ }
+ if len(list) == 0 {
+ return nil, errors.New("policy Statement is null or empty")
+ }
+ for i, st := range list {
+ if st == nil {
+ return nil, fmt.Errorf("statement %d is null", i)
+ }
+ }
+ return list, nil
+}
+
+// conditionValue reports whether a condition value is a string, a number,
+// a boolean, or a list of at least one of those.
+func conditionValue(raw json.RawMessage) bool {
+ var scalar any
+ if err := json.Unmarshal(raw, &scalar); err != nil {
+ return false
+ }
+ switch v := scalar.(type) {
+ case string, float64, bool:
+ return true
+ case []any:
+ if len(v) == 0 {
+ return false
+ }
+ for _, e := range v {
+ switch e.(type) {
+ case string, float64, bool:
+ default:
+ return false
+ }
+ }
+ return true
+ }
+ return false
+}
+
+func statementEffect(i int, st map[string]json.RawMessage) (allow bool, err error) {
+ var effect string
+ if raw, ok := st["Effect"]; !ok || json.Unmarshal(raw, &effect) != nil || (effect != "Allow" && effect != "Deny") {
+ return false, fmt.Errorf("statement %d has the effect %s", i, string(st["Effect"]))
+ }
+ return effect == "Allow", nil
+}
+
+// decodePolicy reads an identity policy. It checks the grammar before it
+// evaluates anything: a shape it does not know is an error, and is never
+// a statement that reaches nothing.
+func decodePolicy(raw string) (*policyDoc, error) {
+ doc, err := policyText(raw)
+ if err != nil {
+ return nil, err
+ }
+ list, err := policyStatements(doc)
+ if err != nil {
+ return nil, err
+ }
+ out := &policyDoc{}
+ identity := map[string]bool{"Sid": true, "Effect": true, "Action": true, "NotAction": true, "Resource": true, "NotResource": true, "Condition": true}
+ for i, st := range list {
+ if err := statementShape(i, st, identity); err != nil {
+ return nil, err
+ }
+ allow, err := statementEffect(i, st)
+ if err != nil {
+ return nil, err
+ }
+ ps := policyStmt{allow: allow}
+ _, hasAction := st["Action"]
+ _, ps.hasNotAction = st["NotAction"]
+ _, hasResource := st["Resource"]
+ _, ps.hasNotResource = st["NotResource"]
+ if hasAction == ps.hasNotAction {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Action and NotAction", i)
+ }
+ if hasResource == ps.hasNotResource {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Resource and NotResource", i)
+ }
+ for _, key := range []string{"Action", "NotAction", "Resource", "NotResource"} {
+ raw, ok := st[key]
+ if !ok {
+ continue
+ }
+ values, err := stringOrList(raw)
+ if err != nil {
+ return nil, fmt.Errorf("statement %d %s: %w", i, key, err)
+ }
+ switch key {
+ case "Action":
+ ps.actions = values
+ case "Resource":
+ ps.resources = values
+ }
+ }
+ // A Deny is validated like an Allow and then not evaluated: it
+ // only narrows.
+ out.statements = append(out.statements, ps)
+ }
+ return out, nil
+}
+
+// trustPrincipals reads a role's trust policy and gives every principal
+// its Allow statements name, sorted. NotPrincipal on an Allow is "*".
+func trustPrincipals(raw string) ([]string, error) {
+ doc, err := policyText(raw)
+ if err != nil {
+ return nil, err
+ }
+ list, err := policyStatements(doc)
+ if err != nil {
+ return nil, err
+ }
+ seen := map[string]bool{}
+ trust := map[string]bool{"Sid": true, "Effect": true, "Principal": true, "NotPrincipal": true, "Action": true, "NotAction": true, "Condition": true}
+ for i, st := range list {
+ // Every statement is read in full, a Deny included: its shape is
+ // the grammar's or the document is not read.
+ if err := statementShape(i, st, trust); err != nil {
+ return nil, err
+ }
+ allow, err := statementEffect(i, st)
+ if err != nil {
+ return nil, err
+ }
+ _, hasAction := st["Action"]
+ _, hasNotAction := st["NotAction"]
+ if hasAction == hasNotAction {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Action and NotAction", i)
+ }
+ for _, key := range []string{"Action", "NotAction"} {
+ if raw, ok := st[key]; ok {
+ if _, err := stringOrList(raw); err != nil {
+ return nil, fmt.Errorf("statement %d %s: %w", i, key, err)
+ }
+ }
+ }
+ rawPrincipal, hasPrincipal := st["Principal"]
+ rawNot, hasNot := st["NotPrincipal"]
+ if hasPrincipal == hasNot {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Principal and NotPrincipal", i)
+ }
+ // The principal element's shape, whichever it is: "*", or a map
+ // of the kinds a trust policy names, each a string or a list of
+ // strings, none empty.
+ principals := map[string]bool{}
+ readPrincipal := func(raw json.RawMessage) error {
+ var star string
+ if json.Unmarshal(raw, &star) == nil {
+ if star != "*" {
+ return fmt.Errorf("statement %d has the principal %q", i, star)
+ }
+ principals["*"] = true
+ return nil
+ }
+ var kinds map[string]json.RawMessage
+ if err := json.Unmarshal(raw, &kinds); err != nil || len(kinds) == 0 {
+ return fmt.Errorf("statement %d has a principal that is neither \"*\" nor a map of kinds", i)
+ }
+ for kind, rawValues := range kinds {
+ switch kind {
+ case "AWS", "Service", "Federated", "CanonicalUser":
+ default:
+ return fmt.Errorf("statement %d has the principal kind %q", i, kind)
+ }
+ values, err := stringOrList(rawValues)
+ if err != nil {
+ return fmt.Errorf("statement %d principal %s: %w", i, kind, err)
+ }
+ for _, v := range values {
+ if v == "*" {
+ principals["*"] = true
+ } else {
+ principals[kind+":"+v] = true
+ }
+ }
+ }
+ return nil
+ }
+ if hasPrincipal {
+ if err := readPrincipal(rawPrincipal); err != nil {
+ return nil, err
+ }
+ } else {
+ if err := readPrincipal(rawNot); err != nil {
+ return nil, err
+ }
+ }
+ if !allow {
+ continue
+ }
+ if hasNot {
+ // Everyone but the named: anyone, as far as this report says.
+ seen["*"] = true
+ continue
+ }
+ for v := range principals {
+ seen[v] = true
+ }
+ }
+ out := make([]string, 0, len(seen))
+ for v := range seen {
+ out = append(out, v)
+ }
+ sort.Strings(out)
+ return out, nil
+}
+
+// possiblyPrefix marks, in a Via entry, a reach through an S3 resource
+// that is not the bucket itself and whose relation to the bucket the
+// report cannot see.
+const possiblyPrefix = "possibly, through an S3 resource that is not the bucket (an access point, an Access Grants instance, a job, a Storage Lens configuration, or anything else S3 or Object Lambda names that is not a general-purpose bucket): "
+
+// reachKind is how a document reaches a bucket: directly (the bucket's
+// ARN or an object under it), or possibly, through any S3 resource that
+// is not a general-purpose bucket or an object in one, whose relation to
+// this bucket the report cannot see (an access point, a multi-region
+// access point, an Object Lambda access point, an Access Grants instance,
+// a job, a Storage Lens configuration, a type it does not know). A
+// possible reach is reported, never dropped.
+type reachKind int
+
+const (
+ reachNone reachKind = iota
+ reachPossible
+ reachDirect
+)
+
+// reaches reports whether any Allow statement of the document can apply
+// an S3 action to the bucket or to an object under it, directly or
+// possibly.
+func (d *policyDoc) reaches(partition, bucket string) reachKind {
+ arn := "arn:" + partition + ":s3:::" + bucket
+ kind := reachNone
+ for _, s := range d.statements {
+ if !s.allow {
+ continue
+ }
+ action, lambda := s.hasNotAction, s.hasNotAction
+ for _, a := range s.actions {
+ if actionCanBeS3(a) {
+ action = true
+ }
+ if actionCanBeObjectLambda(a) {
+ lambda = true
+ }
+ }
+ if !action && !lambda {
+ continue
+ }
+ if s.hasNotResource {
+ if action {
+ return reachDirect
+ }
+ kind = max(kind, reachPossible)
+ continue
+ }
+ for _, r := range s.resources {
+ if action && resourceReaches(r, arn) {
+ return reachDirect
+ }
+ if resourceIsAnAccessPath(r, partition, action, lambda) {
+ kind = max(kind, reachPossible)
+ }
+ }
+ }
+ return kind
+}
+
+// arnFields splits a resource pattern into the six fields of an ARN,
+// filling the fields an incomplete ARN leaves out with stars: one reading
+// of an incomplete pattern (under it "arn:aws:s3" names every S3
+// resource). The other reading is the text itself, a star free to run
+// across colons; the matchers try both, because which reading IAM gives
+// is unverified. A pattern that does not begin "arn:" is returned as nil.
+func arnFields(pattern string) []string {
+ if !strings.HasPrefix(pattern, "arn:") && pattern != "arn" {
+ return nil
+ }
+ parts := strings.SplitN(pattern, ":", 6)
+ for len(parts) < 6 {
+ parts = append(parts, "*")
+ }
+ return parts
+}
+
+// resourceIsAnAccessPath reports whether a resource pattern can name an S3
+// resource that is not a general-purpose bucket or an object in one (an
+// access point, a multi-region access point, an Access Grants instance or
+// grant, a Batch Operations job, a Storage Lens configuration, an Object
+// Lambda access point, a type this report does not know), through which
+// a bucket's objects can be read or written by a relation the ARN does
+// not show (a Storage Lens configuration, for one, exports into a bucket
+// of its owner's choosing). Which bucket is not in the ARN, so any of
+// them is a possible reach of every bucket. One text test decides it, on
+// the pattern and, when it is an incomplete ARN, on its padded form: an
+// S3 resource other than a general-purpose bucket's is one whose ARN has
+// a region or an account, so the pattern must be able to match
+// "arn:<partition>:s3:" or "arn:<partition>:s3::" and then a character
+// that is not a colon (a star or a question mark can stand for that
+// character, so a wildcard that runs across a colon counts); an Object
+// Lambda resource is anything the pattern can name under
+// "arn:<partition>:s3-object-lambda:", under an S3 action (its
+// management actions) or an Object Lambda one. A pattern that cannot reach those
+// prefixes as text (another service's ARN, a bucket's own ARN with its
+// empty region and account, a word) is not an access path.
+func resourceIsAnAccessPath(pattern, partition string, s3, lambda bool) bool {
+ p := conservativePattern(pattern)
+ readings := []string{p}
+ if parts := arnFields(p); parts != nil && strings.Join(parts, ":") != p {
+ readings = append(readings, strings.Join(parts, ":"))
+ }
+ for _, p := range readings {
+ if s3 && (continuesPastAColon(p, "arn:"+partition+":s3:") || continuesPastAColon(p, "arn:"+partition+":s3::")) {
+ return true
+ }
+ // Object Lambda's management actions are in the s3 namespace
+ // (s3:PutAccessPointPolicyForObjectLambda) and its data actions in
+ // its own; either kind on one of its resources is a possible reach.
+ if (s3 || lambda) && wildcardLive(p, "arn:"+partition+":s3-object-lambda:") != nil {
+ return true
+ }
+ }
+ return false
+}
+
+// continuesPastAColon reports whether the pattern can match the prefix and
+// then one more character that is not a colon.
+func continuesPastAColon(pattern, prefix string) bool {
+ p := []rune(pattern)
+ for at := range wildcardLive(pattern, prefix) {
+ if at < len(p) && p[at] != ':' {
+ return true
+ }
+ }
+ return false
+}
+
+// actionCanBeObjectLambda reports whether an action pattern can name an
+// action of the Object Lambda service.
+func actionCanBeObjectLambda(pattern string) bool {
+ p := strings.ToLower(conservativePattern(pattern))
+ service, _, ok := strings.Cut(p, ":")
+ if !ok {
+ return wildcardLive(p, "s3-object-lambda:") != nil
+ }
+ return wildcardMatch(service, "s3-object-lambda")
+}
+
+// actionCanBeS3 reports whether an action pattern can name an action of
+// the S3 service. Action names are not case-sensitive.
+func actionCanBeS3(pattern string) bool {
+ p := strings.ToLower(conservativePattern(pattern))
+ service, _, ok := strings.Cut(p, ":")
+ if !ok {
+ // No service prefix: "*", or something that is not an action; a
+ // pattern that can match "s3:" followed by anything is taken to.
+ return wildcardLive(p, "s3:") != nil
+ }
+ return wildcardMatch(service, "s3")
+}
+
+// resourceReaches reports whether a resource pattern matches the bucket's
+// ARN, or can match the ARN of some object under it, under either reading
+// of the pattern: as text, or as the fields of an ARN with the ones an
+// incomplete pattern leaves out as stars.
+func resourceReaches(pattern, bucketARN string) bool {
+ p := conservativePattern(pattern)
+ readings := []string{p}
+ if parts := arnFields(p); parts != nil && strings.Join(parts, ":") != p {
+ readings = append(readings, strings.Join(parts, ":"))
+ }
+ for _, p := range readings {
+ if resourceReadingReaches(p, bucketARN) {
+ return true
+ }
+ }
+ return false
+}
+
+func resourceReadingReaches(p, bucketARN string) bool {
+ if wildcardMatch(p, bucketARN) {
+ return true
+ }
+ // An object ARN is the bucket's, a slash, and a key of at least one
+ // character. The pattern matches one if, having matched "<arn>/", some
+ // of it is left: what is left of a pattern always matches some string
+ // of at least one character (a star can stand for one). A pattern
+ // used up exactly at the slash matches only the empty key, which no
+ // object has.
+ for at := range wildcardLive(p, bucketARN+"/") {
+ if at < len([]rune(p)) {
+ return true
+ }
+ }
+ return false
+}
+
+// conservativePattern replaces each policy variable with a star: a
+// variable expands to a value this inventory does not know.
+func conservativePattern(pattern string) string {
+ for {
+ i := strings.Index(pattern, "${")
+ if i < 0 {
+ return pattern
+ }
+ j := strings.Index(pattern[i:], "}")
+ if j < 0 {
+ return pattern[:i] + "*"
+ }
+ pattern = pattern[:i] + "*" + pattern[i+j+1:]
+ }
+}
+
+// wildcardLive runs an IAM wildcard pattern (* any run, ? any one
+// character) over text and returns the set of pattern positions it can be
+// at once all of text is consumed, or nil if it cannot consume it.
+func wildcardLive(pattern, text string) map[int]bool {
+ p := []rune(pattern)
+ closure := func(set map[int]bool) {
+ for changed := true; changed; {
+ changed = false
+ for i := range set {
+ if i < len(p) && p[i] == '*' && !set[i+1] {
+ set[i+1] = true
+ changed = true
+ }
+ }
+ }
+ }
+ live := map[int]bool{0: true}
+ closure(live)
+ for _, c := range text {
+ next := map[int]bool{}
+ for i := range live {
+ if i >= len(p) {
+ continue
+ }
+ switch {
+ case p[i] == '*':
+ next[i] = true
+ case p[i] == '?' || p[i] == c:
+ next[i+1] = true
+ }
+ }
+ if len(next) == 0 {
+ return nil
+ }
+ closure(next)
+ live = next
+ }
+ return live
+}
+
+// wildcardMatch reports whether the pattern matches the whole of text.
+func wildcardMatch(pattern, text string) bool {
+ return wildcardLive(pattern, text)[len([]rune(pattern))]
+}
diff --git a/internal/objstore/principals_export_test.go b/internal/objstore/principals_export_test.go
new file mode 100644
index 000000000..d45e255c9
--- /dev/null
+++ b/internal/objstore/principals_export_test.go
@@ -0,0 +1,9 @@
+package objstore
+
+// inventoryMaxPagesForTest sets the listing's page bound and returns the
+// one it replaced.
+func inventoryMaxPagesForTest(n int) int {
+ old := inventoryMaxPages
+ inventoryMaxPages = n
+ return old
+}
diff --git a/internal/objstore/principals_test.go b/internal/objstore/principals_test.go
new file mode 100644
index 000000000..ff47d774e
--- /dev/null
+++ b/internal/objstore/principals_test.go
@@ -0,0 +1,732 @@
+package objstore
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "net/url"
+ "reflect"
+ "strings"
+ "testing"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ "github.com/aws/aws-sdk-go-v2/service/iam"
+ iamtypes "github.com/aws/aws-sdk-go-v2/service/iam/types"
+)
+
+// stubInventory answers GetAccountAuthorizationDetails with fixed pages.
+type stubInventory struct {
+ pages []*iam.GetAccountAuthorizationDetailsOutput
+ err error
+ calls []string // the marker each call carried
+}
+
+func (s *stubInventory) GetAccountAuthorizationDetails(_ context.Context, in *iam.GetAccountAuthorizationDetailsInput, _ ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error) {
+ s.calls = append(s.calls, aws.ToString(in.Marker))
+ if s.err != nil {
+ return nil, s.err
+ }
+ if len(in.Filter) != 0 {
+ return nil, errors.New("the inventory filtered the listing: it must read every kind of entity")
+ }
+ i := len(s.calls) - 1
+ if i >= len(s.pages) {
+ return nil, fmt.Errorf("page %d asked for; there are %d", i, len(s.pages))
+ }
+ return s.pages[i], nil
+}
+
+const invAccount = "arn:aws:iam::123456789012:"
+
+// encoded is a policy document as IAM returns it: percent-encoded.
+func encoded(doc string) *string { return aws.String(url.PathEscape(doc)) }
+
+func allow(action, resource string) string {
+ return fmt.Sprintf(`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":%s,"Resource":%s}]}`, action, resource)
+}
+
+func invUser(name string, groups []string, inline map[string]string, attached ...string) iamtypes.UserDetail {
+ u := iamtypes.UserDetail{Arn: aws.String(invAccount + "user/" + name), UserName: aws.String(name), GroupList: groups}
+ for n, d := range inline {
+ u.UserPolicyList = append(u.UserPolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ u.AttachedManagedPolicies = append(u.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return u
+}
+
+const trustEC2 = `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}`
+
+func invRole(name string, inline map[string]string, attached ...string) iamtypes.RoleDetail {
+ r := iamtypes.RoleDetail{Arn: aws.String(invAccount + "role/" + name), RoleName: aws.String(name), AssumeRolePolicyDocument: encoded(trustEC2)}
+ for n, d := range inline {
+ r.RolePolicyList = append(r.RolePolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ r.AttachedManagedPolicies = append(r.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return r
+}
+
+func invGroup(name string, inline map[string]string, attached ...string) iamtypes.GroupDetail {
+ g := iamtypes.GroupDetail{Arn: aws.String(invAccount + "group/" + name), GroupName: aws.String(name)}
+ for n, d := range inline {
+ g.GroupPolicyList = append(g.GroupPolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ g.AttachedManagedPolicies = append(g.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return g
+}
+
+func invManaged(arn, doc string) iamtypes.ManagedPolicyDetail {
+ return iamtypes.ManagedPolicyDetail{Arn: aws.String(arn), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), Document: encoded(allow(`"s3:*"`, `"*"`))}, // an old version that reaches everything: not the default, not read
+ {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(doc)},
+ }}
+}
+
+// inventoryFor builds the report over fixed pages.
+func inventoryFor(t *testing.T, platform []string, pages ...*iam.GetAccountAuthorizationDetailsOutput) (*PrincipalReport, *stubInventory) {
+ t.Helper()
+ inv := &stubInventory{pages: pages}
+ p, err := NewPrincipalReport(inv, platform)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return p, inv
+}
+
+func arns(list []PrincipalGrant) []string {
+ out := []string{}
+ for _, g := range list {
+ out = append(out, g.ARN)
+ }
+ return out
+}
+
+// The matcher: which resource and action patterns reach a bucket name.
+func TestPolicyPatternsThatReachABucket(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ for _, tc := range []struct {
+ name, action, resource string
+ want bool
+ }{
+ {"the store user's own bucket statement", `["s3:ListBucket"]`, `"arn:aws:s3:::carlos-store-bac-blog"`, true},
+ {"the store user's own object statement", `["s3:GetObject","s3:PutObject"]`, `"arn:aws:s3:::carlos-store-bac-blog/*"`, true},
+ {"a prefix grant on a platform role", `"s3:CreateBucket"`, `"arn:aws:s3:::carlos-store-*"`, true},
+ {"a prefix grant on objects", `"s3:DeleteObject"`, `"arn:aws:s3:::carlos-*/*"`, true},
+ {"everything", `"*"`, `"*"`, true},
+ {"all of s3 on everything", `"s3:*"`, `"*"`, true},
+ {"a partial action pattern", `"s3:Get*"`, `"arn:aws:s3:::*"`, true},
+ {"an action in another case", `"S3:GETOBJECT"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`, true},
+ {"a question mark standing for one character", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blo?/*"`, true},
+ {"one object key", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/exports/2026.csv"`, true},
+ {"a policy variable", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-${aws:username}/*"`, true},
+ {"a policy variable left open", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-${aws:username"`, true},
+ {"any partition, region and account fields", `"s3:GetObject"`, `"arn:*:s3:*:*:carlos-store-bac-blog/*"`, true},
+ {"one of several resources", `"s3:GetObject"`, `["arn:aws:s3:::other","arn:aws:s3:::carlos-store-bac-blog/*"]`, true},
+ {"one of several actions", `["ec2:RunInstances","s3:ListBucket"]`, `"arn:aws:s3:::carlos-store-bac-blog"`, true},
+ {"a wildcard service", `"s*:List*"`, `"*"`, true},
+
+ {"another bucket", `"s3:*"`, `"arn:aws:s3:::carlos-store-bad-blog"`, false},
+ {"a bucket whose name this one begins", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac"`, false},
+ {"the objects of a bucket whose name this one begins", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac/*"`, false},
+ {"a bucket whose name begins with this one", `"s3:*"`, `["arn:aws:s3:::carlos-store-bac-blog2","arn:aws:s3:::carlos-store-bac-blog2/*"]`, false},
+ {"a question mark that needs one more character", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog?"`, false},
+ {"the slash and nothing after it", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog/"`, false},
+ {"a star after the name, which a slash and a key satisfy", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog*"`, true},
+ {"a question mark for the slash, then a star", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog?*"`, true},
+ {"another service on everything", `"ec2:*"`, `"*"`, false},
+ {"another service whose name begins s3", `"s3-object-lambda:*"`, `"*"`, false},
+ {"another partition", `"s3:*"`, `"arn:aws-cn:s3:::carlos-store-bac-blog"`, false},
+ } {
+ doc, err := decodePolicy(allow(tc.action, tc.resource))
+ if err != nil {
+ t.Errorf("%s: %v", tc.name, err)
+ continue
+ }
+ if got := doc.reaches("aws", bucket) == reachDirect; got != tc.want {
+ t.Errorf("%s: reaches %v, want %v", tc.name, got, tc.want)
+ }
+ }
+ for name, tc := range map[string]struct {
+ doc string
+ want bool
+ }{
+ "NotResource on an Allow": {`{"Statement":[{"Effect":"Allow","Action":"s3:GetObject","NotResource":"arn:aws:s3:::carlos-store-bac-blog/*"}]}`, true},
+ "NotAction on an Allow": {`{"Statement":[{"Effect":"Allow","NotAction":"s3:*","Resource":"*"}]}`, true},
+ "NotAction with another bucket": {`{"Statement":[{"Effect":"Allow","NotAction":"iam:*","Resource":"arn:aws:s3:::other"}]}`, false},
+ "a Deny alone": {`{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, false},
+ "an Allow that a Deny takes back": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"},{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, true},
+ "an Allow under a condition": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"Bool":{"aws:MultiFactorAuthPresent":"true"}}}]}`, true},
+ "one statement, not a list": {`{"Statement":{"Effect":"Allow","Action":"s3:*","Resource":"*"}}`, true},
+ "the older version": {`{"Version":"2008-10-17","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},
+ "a condition with a list of scalars": {`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"StringEquals":{"aws:username":["a","b"]},"Bool":{"aws:SecureTransport":true},"NumericLessThan":{"s3:max-keys":10}}}]}`, true},
+ "a statement id and a policy id": {`{"Version":"2012-10-17","Id":"p","Statement":[{"Sid":"one","Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},
+ "an escaped star, taken as any": {`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:GetObject","Resource":"arn:aws:s3:::carlos-store-bac-${*}/*"}]}`, true},
+ "the second of two statements": {`{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"},{"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::carlos-store-bac-blog"}]}`, true},
+ } {
+ doc, err := decodePolicy(tc.doc)
+ if err != nil {
+ t.Errorf("%s: %v", name, err)
+ continue
+ }
+ if got := doc.reaches("aws", bucket) == reachDirect; got != tc.want {
+ t.Errorf("%s: reaches %v, want %v", name, got, tc.want)
+ }
+ }
+ // What it cannot read is an error, never "does not reach".
+ for name, doc := range map[string]string{
+ "an unknown effect": `{"Statement":[{"Effect":"Permit","Action":"s3:*","Resource":"*"}]}`,
+ "an effect in the wrong case": `{"Statement":[{"Effect":"allow","Action":"s3:*","Resource":"*"}]}`,
+ "an Allow with no resource": `{"Statement":[{"Effect":"Allow","Action":"s3:*"}]}`,
+ "an Allow with no action": `{"Statement":[{"Effect":"Allow","Resource":"*"}]}`,
+ "both Resource and NotResource": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","NotResource":"*"}]}`,
+ "a resource that is a number": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":7}]}`,
+ "a statement that is null": `{"Statement":null}`,
+ "no statements": `{"Statement":[]}`,
+ "a null among the statements": `{"Statement":[null]}`,
+ "no actions": `{"Statement":[{"Effect":"Allow","Action":[],"Resource":"*"}]}`,
+ "no resources": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":[]}]}`,
+ "an action that is null": `{"Statement":[{"Effect":"Allow","Action":null,"Resource":"*"}]}`,
+ "a null among the resources": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":["arn:aws:s3:::x",null]}]}`,
+ "an empty resource": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":""}]}`,
+ "a principal, as a bucket policy has": `{"Statement":[{"Effect":"Allow","Principal":"*","Action":"s3:*","Resource":"*"}]}`,
+ "a NotPrincipal": `{"Statement":[{"Effect":"Deny","NotPrincipal":{"AWS":"*"},"Action":"s3:*","Resource":"*"}]}`,
+ "an element it does not know": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Scope":"all"}]}`,
+ "a version it does not know": `{"Version":"2030-01-01","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]}`,
+ "a version that is a number": `{"Version":2012,"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]}`,
+ "a top-level element it does not know": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}],"Extra":1}`,
+ "a Deny with no resource": `{"Statement":[{"Effect":"Deny","Action":"s3:*"}]}`,
+ "a document that is a list": `[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]`,
+ "a member given twice, the second harmless": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}],"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,
+ "a member given twice inside a statement": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Action":"ec2:*","Resource":"*"}]}`,
+ "a condition that is a number": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":7}]}`,
+ "a condition that is a list": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":["x"]}]}`,
+ "an id that is null": `{"Id":null,"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,
+ "a sid that is null": `{"Statement":[{"Sid":null,"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,
+ "a condition operator that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":null}}]}`,
+ "a condition value that is an object": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":{"unexpected":"object"}}}}]}`,
+ "a condition value that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":null}}}]}`,
+ "an empty condition": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{}}]}`,
+ "a condition operator with no key": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{}}}]}`,
+ "a condition value that is an empty list": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":[]}}}]}`,
+ "a Deny with an empty condition": `{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*","Condition":{}}]}`,
+ "a sid that is a number": `{"Statement":[{"Sid":1,"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,
+ "an id that is a number": `{"Id":1,"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,
+ "a Deny with an unknown element": `{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*","Unexpected":true}]}`,
+ "a Deny with a null action": `{"Statement":[{"Effect":"Deny","Action":null,"Resource":"*"}]}`,
+ "null": `null`,
+ "no statement": `{"Version":"2012-10-17"}`,
+ "not JSON": `Statement: none`,
+ "an empty document": ``,
+ "a broken percent-encoding": `%7B%2`,
+ "a statement that is a string": `{"Statement":"Allow"}`,
+ } {
+ if _, err := decodePolicy(doc); err == nil {
+ t.Errorf("%s: decoded; want an error", name)
+ }
+ }
+ // A plus sign in an encoded document is a plus sign, not a space.
+ doc, err := decodePolicy(`%7B%22Statement%22%3A%5B%7B%22Effect%22%3A%22Allow%22%2C%22Action%22%3A%22s3%3A%2A%22%2C%22Resource%22%3A%22arn%3Aaws%3As3%3A%3A%3Aa+b%22%7D%5D%7D`)
+ if err != nil || doc.reaches("aws", "a+b") != reachDirect || doc.reaches("aws", "a b") != reachNone {
+ t.Errorf("an encoded plus sign: %v", err)
+ }
+}
+
+// The inventory over an account: every kind of principal, inline and
+// managed, direct and through a group; the platform's declared own apart.
+func TestInventoryListsEveryPrincipalThatReachesTheName(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ const storeARN = `"arn:aws:s3:::carlos-store-bac-blog"`
+ const managedRead = "arn:aws:iam::123456789012:policy/store-readers"
+ const managedOther = "arn:aws:iam::123456789012:policy/ec2-only"
+ const awsAdmin = "arn:aws:iam::aws:policy/AdministratorAccess"
+ page1 := &iam.GetAccountAuthorizationDetailsOutput{
+ IsTruncated: true, Marker: aws.String("m1"),
+ UserDetailList: []iamtypes.UserDetail{
+ invUser("carlos-store-bac-blog", nil, map[string]string{"carlos-store": allow(`"s3:ListBucket"`, storeARN)}),
+ invUser("carlos-store-bad-shop", nil, map[string]string{"carlos-store": allow(`"s3:ListBucket"`, `"arn:aws:s3:::carlos-store-bad-shop"`)}),
+ invUser("in-a-reading-group", []string{"readers"}, nil),
+ invUser("in-an-idle-group", []string{"idle"}, nil),
+ invUser("with-a-managed-policy", nil, nil, managedRead),
+ invUser("with-another-managed-policy", nil, nil, managedOther),
+ },
+ GroupDetailList: []iamtypes.GroupDetail{
+ invGroup("readers", nil, managedRead),
+ invGroup("idle", map[string]string{"ec2": allow(`"ec2:*"`, `"*"`)}),
+ },
+ }
+ page2 := &iam.GetAccountAuthorizationDetailsOutput{
+ RoleDetailList: []iamtypes.RoleDetail{
+ invRole("carlos-flagship-host", map[string]string{"carlos-flagship-store": allow(`["s3:CreateBucket","s3:DeleteBucket"]`, `"arn:aws:s3:::carlos-store-*"`)}),
+ invRole("OrganizationAccountAccessRole", nil, awsAdmin),
+ invRole("lambda-thumbnails", map[string]string{"own": allow(`"s3:*"`, `["arn:aws:s3:::thumbnails","arn:aws:s3:::thumbnails/*"]`)}),
+ },
+ Policies: []iamtypes.ManagedPolicyDetail{
+ invManaged(managedRead, allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-*/*"`)),
+ invManaged(managedOther, allow(`"ec2:*"`, `"*"`)),
+ invManaged(awsAdmin, allow(`"*"`, `"*"`)),
+ },
+ }
+ p, stub := inventoryFor(t, nil, page1, page2)
+ inv, err := p.BucketPrincipals(context.Background(), bucket)
+ if err != nil {
+ t.Fatal(err)
+ }
+ want := []string{
+ invAccount + "group/readers",
+ invAccount + "role/OrganizationAccountAccessRole",
+ invAccount + "role/carlos-flagship-host",
+ invAccount + "user/carlos-store-bac-blog",
+ invAccount + "user/in-a-reading-group",
+ invAccount + "user/with-a-managed-policy",
+ }
+ if got := arns(inv.Reaching); !reflect.DeepEqual(got, want) {
+ t.Fatalf("reaching\n %v\nwant\n %v", got, want)
+ }
+ if len(inv.NamedByDeployment) != 0 || inv.Users != 6 || inv.Groups != 2 || inv.Roles != 3 || inv.ManagedPolicies != 3 {
+ t.Fatalf("inventory %+v", inv)
+ }
+ if !reflect.DeepEqual(stub.calls, []string{"", "m1"}) {
+ t.Fatalf("pages asked for: %q; want the first, then the marker", stub.calls)
+ }
+ via := map[string][]string{}
+ for _, g := range inv.Reaching {
+ via[g.ARN] = g.Via
+ if (g.Kind == "role") != reflect.DeepEqual(g.AssumableBy, []string{"Service:ec2.amazonaws.com"}) {
+ t.Errorf("%s: assumable by %v", g.ARN, g.AssumableBy)
+ }
+ if wantKind := strings.SplitN(strings.TrimPrefix(g.ARN, invAccount), "/", 2)[0]; g.Kind != wantKind {
+ t.Errorf("%s has kind %q", g.ARN, g.Kind)
+ }
+ }
+ for arn, want := range map[string][]string{
+ invAccount + "user/carlos-store-bac-blog": {"inline:carlos-store"},
+ invAccount + "user/in-a-reading-group": {"group:" + invAccount + "group/readers"},
+ invAccount + "user/with-a-managed-policy": {"managed:" + managedRead},
+ invAccount + "group/readers": {"managed:" + managedRead},
+ invAccount + "role/carlos-flagship-host": {"inline:carlos-flagship-store"},
+ invAccount + "role/OrganizationAccountAccessRole": {"managed:" + awsAdmin},
+ } {
+ if !reflect.DeepEqual(via[arn], want) {
+ t.Errorf("%s reaches through %v; want %v", arn, via[arn], want)
+ }
+ }
+ // With the platform's own named, they are set apart; that is a label,
+ // and they are still in the report.
+ platform := []string{invAccount + "role/carlos-flagship-host", invAccount + "role/OrganizationAccountAccessRole", invAccount + "role/not-in-the-account"}
+ p, _ = inventoryFor(t, platform, page1, page2)
+ inv, err = p.BucketPrincipals(context.Background(), bucket)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if got := arns(inv.NamedByDeployment); !reflect.DeepEqual(got, []string{platform[1], platform[0]}) {
+ t.Fatalf("platform %v", got)
+ }
+ if got := arns(inv.Reaching); !reflect.DeepEqual(got, []string{want[0], want[3], want[4], want[5]}) {
+ t.Fatalf("reaching with the platform named: %v", got)
+ }
+ p, _ = inventoryFor(t, platform, page1, page2)
+ if inv, err := p.BucketPrincipals(context.Background(), "carlos-store-new-name"); err != nil || !reflect.DeepEqual(arns(inv.Reaching), []string{invAccount + "group/readers", invAccount + "user/in-a-reading-group", invAccount + "user/with-a-managed-policy"}) {
+ t.Fatalf("a new name under the prefix: %v %v; want the prefix readers, who are nobody's named own", inv, err)
+ }
+ p, _ = inventoryFor(t, platform, page1, page2)
+ if inv, err := p.BucketPrincipals(context.Background(), "elsewhere-entirely"); err != nil || len(inv.Reaching) != 0 || len(inv.NamedByDeployment) != 1 {
+ t.Fatalf("a name only the administrator reaches: %+v %v", inv, err)
+ }
+}
+
+// A role is reported with the principal values its trust policy's Allow
+// statements name: another account, a federation, anyone; not who can in
+// fact assume it.
+func TestInventoryReportsWhoARoleTrusts(t *testing.T) {
+ role := func(name, trust string) iamtypes.RoleDetail {
+ r := invRole(name, map[string]string{"p": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)})
+ r.AssumeRolePolicyDocument = encoded(trust)
+ return r
+ }
+ trust := func(principal string) string {
+ return `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":` + principal + `,"Action":"sts:AssumeRole"}]}`
+ }
+ p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{
+ role("a-another-account", trust(`{"AWS":"arn:aws:iam::999999999999:root"}`)),
+ role("b-several", trust(`{"AWS":["arn:aws:iam::123456789012:user/ops","111111111111"],"Service":"lambda.amazonaws.com"}`)),
+ role("c-anyone", trust(`"*"`)),
+ role("d-anyone-in-a-map", trust(`{"AWS":"*"}`)),
+ role("e-federated", trust(`{"Federated":"arn:aws:iam::123456789012:saml-provider/idp"}`)),
+ role("f-not-principal", `{"Statement":[{"Effect":"Allow","NotPrincipal":{"AWS":"arn:aws:iam::123456789012:user/x"},"Action":"sts:AssumeRole"},{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole"}]}`),
+ }})
+ inv, err := p.BucketPrincipals(context.Background(), "carlos-store-bac-blog")
+ if err != nil {
+ t.Fatal(err)
+ }
+ want := [][]string{
+ {"AWS:arn:aws:iam::999999999999:root"},
+ {"AWS:111111111111", "AWS:arn:aws:iam::123456789012:user/ops", "Service:lambda.amazonaws.com"},
+ {"*"}, {"*"},
+ {"Federated:arn:aws:iam::123456789012:saml-provider/idp"},
+ {"*"},
+ }
+ if len(inv.Reaching) != len(want) {
+ t.Fatalf("reaching %v", arns(inv.Reaching))
+ }
+ for i, g := range inv.Reaching {
+ if !reflect.DeepEqual(g.AssumableBy, want[i]) {
+ t.Errorf("%s is assumable by %v; want %v", g.ARN, g.AssumableBy, want[i])
+ }
+ }
+ // A trust policy it cannot read is an error, not a role with nobody.
+ for name, doc := range map[string]string{
+ "no principal": `{"Statement":[{"Effect":"Allow","Action":"sts:AssumeRole"}]}`,
+ "a principal that is a word": `{"Statement":[{"Effect":"Allow","Principal":"everyone","Action":"sts:AssumeRole"}]}`,
+ "an empty principal map": `{"Statement":[{"Effect":"Allow","Principal":{},"Action":"sts:AssumeRole"}]}`,
+ "a null principal value": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":null},"Action":"sts:AssumeRole"}]}`,
+ "no trust policy": ``,
+ "an unknown effect": `{"Statement":[{"Effect":"Maybe","Principal":"*"}]}`,
+ "a Deny with an unknown element": `{"Statement":{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Unexpected":true}}`,
+ "a Deny with a null principal": `{"Statement":{"Effect":"Deny","Principal":null,"Action":"sts:AssumeRole"}}`,
+ "an Allow with a null action": `{"Statement":[{"Effect":"Allow","Principal":"*","Action":null}]}`,
+ "an Allow with no action": `{"Statement":[{"Effect":"Allow","Principal":"*"}]}`,
+ "an unknown principal kind": `{"Statement":[{"Effect":"Allow","Principal":{"Anyone":"*"},"Action":"sts:AssumeRole"}]}`,
+ "a NotPrincipal that is null": `{"Statement":[{"Effect":"Allow","NotPrincipal":null,"Action":"sts:AssumeRole"}]}`,
+ "both Principal and NotPrincipal": `{"Statement":[{"Effect":"Allow","Principal":"*","NotPrincipal":{"AWS":"x"},"Action":"sts:AssumeRole"}]}`,
+ "a member given twice": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::999999999999:root"},"Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}`,
+ "a Deny with a malformed condition": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{"StringEquals":{"aws:username":{"x":1}}}}]}`,
+ "a Deny with an empty condition": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{}}]}`,
+ "a Deny whose operator has no key": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{"StringEquals":{}}}]}`,
+ } {
+ p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{role("r", doc)}})
+ if inv, err := p.BucketPrincipals(context.Background(), "carlos-store-bac-blog"); err == nil {
+ t.Errorf("%s: %+v with no error", name, inv)
+ }
+ }
+}
+
+// A grant on any S3 resource that is not a general-purpose bucket or an
+// object in one (an access point, a multi-region access point, an Object
+// Lambda access point, an Access Grants instance, a job, a Storage Lens
+// configuration) names no bucket and can reach a bucket's objects by a
+// relation the ARN does not show: its principal is reported as possibly
+// reaching, marked so. A grant on another service's resource is not,
+// unless, read as text, its wildcards can spell an S3 ARN. A pattern is
+// read as text and as padded fields, and a reach under either is
+// reported.
+func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ for name, tc := range map[string]struct {
+ doc string
+ want reachKind
+ }{
+ "an access point's objects": {allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`), reachPossible},
+ "an access point itself": {allow(`"s3:*"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers"`), reachPossible},
+ // (A star in the account field can, read as text, run into the
+ // resource, so this pattern also reaches an object whose key holds
+ // ":accesspoint/": a direct reach, which is the stronger answer.)
+ "any access point": {allow(`"s3:GetObject"`, `"arn:aws:s3:*:*:accesspoint/*"`), reachDirect},
+ "a multi-region access point": {allow(`"s3:GetObject"`, `"arn:aws:s3::123456789012:accesspoint/mfzwi23gnjvgw.mrap/object/*"`), reachPossible},
+ "an Object Lambda access point": {allow(`"s3-object-lambda:GetObject"`, `"arn:aws:s3-object-lambda:eu-west-1:123456789012:accesspoint/thumbs"`), reachPossible},
+ // Object Lambda's management actions are in the s3 namespace.
+ "an Object Lambda access point's policy": {allow(`"s3:PutAccessPointPolicyForObjectLambda"`, `"arn:aws:s3-object-lambda:eu-west-1:123456789012:accesspoint/readers"`), reachPossible},
+ "all of s3 on an Object Lambda access point": {allow(`"s3:*"`, `"arn:aws:s3-object-lambda:eu-west-1:123456789012:accesspoint/readers"`), reachPossible},
+ "Object Lambda on everything": {allow(`"s3-object-lambda:*"`, `"*"`), reachPossible},
+ "the bucket itself beside an access point": {allow(`"s3:GetObject"`, `["arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*","arn:aws:s3:::carlos-store-bac-blog/*"]`), reachDirect},
+ "another service's resource": {allow(`"s3:GetObject"`, `"arn:aws:lambda:eu-west-1:123456789012:function:thumbs"`), reachNone},
+ "an S3 action on an access point in another partition": {allow(`"s3:GetObject"`, `"arn:aws-cn:s3:cn-north-1:123456789012:accesspoint/readers/object/*"`), reachNone},
+ "an EC2 action on an access point ARN": {allow(`"ec2:*"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers"`), reachNone},
+ "S3 on Outposts": {allow(`"s3-outposts:GetObject"`, `"arn:aws:s3-outposts:eu-west-1:123456789012:outpost/op-1/accesspoint/ap/object/*"`), reachNone},
+ "an Access Grants instance": {allow(`"s3:GetDataAccess"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default"`), reachPossible},
+ "an Access Grants grant": {allow(`"s3:CreateAccessGrant"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default/grant/*"`), reachPossible},
+ "a Batch Operations job": {allow(`"s3:CreateJob"`, `"arn:aws:s3:eu-west-1:123456789012:job/*"`), reachPossible},
+ // (A Storage Lens configuration exports into a bucket of its
+ // owner's choosing: whoever may write one may have objects written.)
+ "reading a Storage Lens configuration": {allow(`"s3:GetStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/x"`), reachPossible},
+ "writing a Storage Lens configuration": {allow(`"s3:PutStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/export"`), reachPossible},
+ "a Storage Lens group": {allow(`"s3:*"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens-group/g"`), reachPossible},
+ "an S3 resource type it does not know": {allow(`"s3:*"`, `"arn:aws:s3:eu-west-1:123456789012:something-new/x"`), reachPossible},
+ "an incomplete ARN for all of S3": {allow(`"s3:GetObject"`, `"arn:aws:s3"`), reachDirect},
+ "an incomplete ARN for all of Object Lambda": {allow(`"s3-object-lambda:GetObject"`, `"arn:aws:s3-object-lambda"`), reachPossible},
+ "an incomplete ARN for another service": {allow(`"s3:GetObject"`, `"arn:aws:lambda"`), reachNone},
+ // Read as padded fields these name a service called by the
+ // bucket's name; read as text the star runs across the fields to
+ // the bucket itself. Either reading reports.
+ "a star that runs across the fields to the bucket": {allow(`"s3:PutBucketPolicy"`, `"arn:*:carlos-store-bac-blog"`), reachDirect},
+ "a star that runs across the fields to a prefix": {allow(`"s3:PutBucketPolicy"`, `"arn:*:carlos-store-*"`), reachDirect},
+ "a star that runs into the bucket's name": {allow(`"s3:PutBucketPolicy"`, `"arn:aws:*carlos-store-bac-blog"`), reachDirect},
+ "a star that runs across the fields to an access point": {allow(`"s3-object-lambda:GetObject"`, `"arn:*:accesspoint/thumbs"`), reachPossible},
+ // (And, read as text, an incomplete ARN of another service whose
+ // star can run into an object key holding ":function:thumbs": a
+ // direct reach, the over-report the text reading costs.)
+ "an incomplete ARN of another service with a star": {allow(`"s3:GetObject"`, `"arn:*:function:thumbs"`), reachDirect},
+ // A question mark can stand for a colon too, and it cannot run
+ // into this bucket's keys the way a star can: these are possible
+ // reaches that neither the field reading nor the direct matcher
+ // sees.
+ "a question mark for the colon after the service": {allow(`"s3:GetObject"`, `"arn:aws:s3?eu-west-1:123456789012:accesspoint/readers/object/a:b"`), reachPossible},
+ "a question mark for the colon, an incomplete pattern": {allow(`"s3:GetObject"`, `"arn:aws:s3?eu-west-1"`), reachPossible},
+ "a question mark for the first letter of the ARN": {allow(`"s3:PutAccessPointPolicy"`, `"?rn:aws:s3:eu-west-1:123456789012:accesspoint/readers"`), reachPossible},
+ "a word with a star, which spells no ARN": {allow(`"s3:GetObject"`, `"readers*"`), reachNone},
+ "a star for everything before the resource": {allow(`"s3:GetObject"`, `"*:accesspoint/readers"`), reachDirect},
+ } {
+ doc, err := decodePolicy(tc.doc)
+ if err != nil {
+ t.Errorf("%s: %v", name, err)
+ continue
+ }
+ if got := doc.reaches("aws", bucket); got != tc.want {
+ t.Errorf("%s: reaches %v, want %v", name, got, tc.want)
+ }
+ }
+ // In the report: set apart in the Via, never dropped; and a member of
+ // a group whose only reach is possible reaches possibly too.
+ p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{
+ UserDetailList: []iamtypes.UserDetail{
+ invUser("through-an-access-point", nil, map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}),
+ invUser("in-the-ap-group", []string{"ap-readers"}, nil),
+ invUser("in-the-direct-group", []string{"direct-readers"}, nil),
+ invUser("in-the-mixed-group", []string{"mixed-readers"}, nil),
+ invUser("over-an-object-lambda-policy", nil, map[string]string{"ol": allow(`"s3:PutAccessPointPolicyForObjectLambda"`, `"arn:aws:s3-object-lambda:eu-west-1:123456789012:accesspoint/readers"`)}),
+ },
+ GroupDetailList: []iamtypes.GroupDetail{
+ invGroup("ap-readers", map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}),
+ invGroup("direct-readers", map[string]string{"d": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)}),
+ invGroup("mixed-readers", map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`), "d": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)}),
+ },
+ })
+ inv, err := p.BucketPrincipals(context.Background(), bucket)
+ if err != nil {
+ t.Fatal(err)
+ }
+ via := map[string][]string{}
+ for _, g := range inv.Reaching {
+ via[g.ARN] = g.Via
+ }
+ if !reflect.DeepEqual(via[invAccount+"user/through-an-access-point"], []string{possiblyPrefix + "inline:ap"}) ||
+ !reflect.DeepEqual(via[invAccount+"user/in-the-ap-group"], []string{possiblyPrefix + "group:" + invAccount + "group/ap-readers"}) ||
+ !reflect.DeepEqual(via[invAccount+"user/in-the-direct-group"], []string{"group:" + invAccount + "group/direct-readers"}) ||
+ !reflect.DeepEqual(via[invAccount+"user/in-the-mixed-group"], []string{"group:" + invAccount + "group/mixed-readers"}) ||
+ !reflect.DeepEqual(via[invAccount+"user/over-an-object-lambda-policy"], []string{possiblyPrefix + "inline:ol"}) {
+ t.Fatalf("the report: %v", via)
+ }
+}
+
+// A principal's policies are matched in the principal's own partition: a
+// bucket ARN is spelled with it.
+func TestInventoryMatchesInThePrincipalsPartition(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ cn := iamtypes.RoleDetail{Arn: aws.String("arn:aws-cn:iam::123456789012:role/reader"), RoleName: aws.String("reader"), AssumeRolePolicyDocument: encoded(trustEC2),
+ RolePolicyList: []iamtypes.PolicyDetail{{PolicyName: aws.String("p"), PolicyDocument: encoded(allow(`"s3:GetObject"`, `"arn:aws-cn:s3:::carlos-store-bac-blog/*"`))}}}
+ other := invRole("spelled-for-another-partition", map[string]string{"p": allow(`"s3:GetObject"`, `"arn:aws-cn:s3:::carlos-store-bac-blog/*"`)})
+ p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{cn, other}})
+ inv, err := p.BucketPrincipals(context.Background(), bucket)
+ if err != nil || !reflect.DeepEqual(arns(inv.Reaching), []string{"arn:aws-cn:iam::123456789012:role/reader"}) {
+ t.Fatalf("reaching %v %v; want the role whose partition its policy is spelled in, and not the one it is not", inv, err)
+ }
+}
+
+// What the report cannot read is an error with its reason, never a
+// shorter list, and it stops asking where it should.
+func TestInventoryClaimsNothingItCannotRead(t *testing.T) {
+ ctx := context.Background()
+ const bucket = "carlos-store-bac-blog"
+ good := func() *iam.GetAccountAuthorizationDetailsOutput {
+ return &iam.GetAccountAuthorizationDetailsOutput{UserDetailList: []iamtypes.UserDetail{invUser("someone", nil, map[string]string{"p": allow(`"ec2:*"`, `"*"`)})}}
+ }
+ reaches := allow(`"s3:*"`, `"*"`)
+ harmless := allow(`"ec2:*"`, `"*"`)
+
+ // The control: the same account, read to the end, reaches nothing.
+ p, stub := inventoryFor(t, nil, good())
+ if inv, err := p.BucketPrincipals(ctx, bucket); err != nil || len(inv.Reaching) != 0 || inv.Users != 1 || len(stub.calls) != 1 {
+ t.Fatalf("control: %+v %v after %d calls", inv, err, len(stub.calls))
+ }
+ // The listing refused (the role may not make it).
+ p, stub = inventoryFor(t, nil, good())
+ stub.err = &smithyAPIErrorStub{code: "AccessDenied"}
+ if inv, err := p.BucketPrincipals(ctx, bucket); err == nil || inv != nil {
+ t.Fatalf("a refused listing: %v %v", inv, err)
+ }
+ // Truncated with no marker: refused at the first page, for that
+ // reason, with no second call.
+ p, stub = inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{IsTruncated: true}, good())
+ if _, err := p.BucketPrincipals(ctx, bucket); !errors.Is(err, errInventoryTruncated) || len(stub.calls) != 1 {
+ t.Fatalf("truncated with no marker: %v after %d calls; want errInventoryTruncated after one", err, len(stub.calls))
+ }
+ // A marker that repeats: refused when it repeats, with no third call
+ // (the third page, which would end the listing cleanly, is never
+ // asked for).
+ again := func() *iam.GetAccountAuthorizationDetailsOutput {
+ return &iam.GetAccountAuthorizationDetailsOutput{IsTruncated: true, Marker: aws.String("m")}
+ }
+ p, stub = inventoryFor(t, nil, again(), again(), good())
+ if _, err := p.BucketPrincipals(ctx, bucket); !errors.Is(err, errInventoryTruncated) || len(stub.calls) != 2 {
+ t.Fatalf("a marker that repeats: %v after %d calls; want errInventoryTruncated after two", err, len(stub.calls))
+ }
+ // A listing that does not end is given up on at the bound, exactly.
+ saved := inventoryMaxPagesForTest(5)
+ endless := &endlessInventory{}
+ rep, _ := NewPrincipalReport(endless, nil)
+ _, err := rep.BucketPrincipals(ctx, bucket)
+ inventoryMaxPagesForTest(saved)
+ if !errors.Is(err, errInventoryEndless) || endless.calls != 5 {
+ t.Fatalf("an endless listing: %v after %d pages; want errInventoryEndless after five", err, endless.calls)
+ }
+
+ // The same thing returned twice is refused, whichever copy reaches.
+ twoPages := func(first, second *iam.GetAccountAuthorizationDetailsOutput) []*iam.GetAccountAuthorizationDetailsOutput {
+ first.IsTruncated, first.Marker = true, aws.String("m1")
+ return []*iam.GetAccountAuthorizationDetailsOutput{first, second}
+ }
+ managed := "arn:aws:iam::123456789012:policy/p"
+ for name, pages := range map[string][]*iam.GetAccountAuthorizationDetailsOutput{
+ "a managed policy on two pages, the first reaching": twoPages(
+ &iam.GetAccountAuthorizationDetailsOutput{Policies: []iamtypes.ManagedPolicyDetail{invManaged(managed, reaches)}, RoleDetailList: []iamtypes.RoleDetail{invRole("r", nil, managed)}},
+ &iam.GetAccountAuthorizationDetailsOutput{Policies: []iamtypes.ManagedPolicyDetail{invManaged(managed, harmless)}}),
+ "a role on two pages, the first reaching": twoPages(
+ &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{invRole("r", map[string]string{"p": reaches})}},
+ &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{invRole("r", map[string]string{"p": harmless})}}),
+ "a user twice on one page": {{UserDetailList: []iamtypes.UserDetail{invUser("u", nil, map[string]string{"p": reaches}), invUser("u", nil, map[string]string{"p": harmless})}}},
+ "a group twice": {{GroupDetailList: []iamtypes.GroupDetail{invGroup("g", nil), invGroup("g", nil)}}},
+ "an inline policy name twice": {{UserDetailList: []iamtypes.UserDetail{{Arn: aws.String(invAccount + "user/u"), UserName: aws.String("u"), UserPolicyList: []iamtypes.PolicyDetail{
+ {PolicyName: aws.String("p"), PolicyDocument: encoded(reaches)}, {PolicyName: aws.String("p"), PolicyDocument: encoded(harmless)}}}}}},
+ "a managed policy whose default version has no id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{{IsDefaultVersion: true, Document: encoded(harmless)}}}}}},
+ "one version id twice": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v1"), Document: encoded(reaches)}}}}}},
+ "a version with no id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {Document: encoded(reaches)}}}}}},
+ } {
+ p, _ := inventoryFor(t, nil, pages...)
+ if inv, err := p.BucketPrincipals(ctx, bucket); !errors.Is(err, errInventoryRepeated) {
+ t.Errorf("%s: %+v %v; want errInventoryRepeated", name, inv, err)
+ }
+ }
+ // The two-default refusal is that refusal, not a later one.
+ twoDefaults := &iam.GetAccountAuthorizationDetailsOutput{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}
+ p, _ = inventoryFor(t, nil, twoDefaults)
+ if _, err := p.BucketPrincipals(ctx, bucket); err == nil || !strings.Contains(err.Error(), "default versions") {
+ t.Fatalf("two default versions: %v; want that refusal by name", err)
+ }
+ for name, pages := range map[string][]*iam.GetAccountAuthorizationDetailsOutput{
+ "a managed policy attached and not listed": {{RoleDetailList: []iamtypes.RoleDetail{invRole("r", nil, "arn:aws:iam::123456789012:policy/missing")}}},
+ "a managed policy with no default version": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), Document: encoded(harmless)}}}}}},
+ "a managed policy whose default id names another version": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), Document: encoded(reaches)}}}}}},
+ "a managed policy with no default id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},
+ "a managed policy with two default versions": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},
+ "two default versions, the first with no document": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},
+ "a default version with no document": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), IsDefaultVersion: true}}}}}},
+ "an inline policy that does not decode": {{UserDetailList: []iamtypes.UserDetail{invUser("u", nil, map[string]string{"p": `{"Statement":[{"Effect":"Allow","Action":"s3:*"}]}`})}}},
+ "an inline policy whose statement is null": {{UserDetailList: []iamtypes.UserDetail{invUser("u", nil, map[string]string{"p": `{"Statement":null}`})}}},
+ "a role's inline policy that does not decode": {{RoleDetailList: []iamtypes.RoleDetail{invRole("r", map[string]string{"p": `not a policy`})}}},
+ "a group's inline policy that does not decode": {{GroupDetailList: []iamtypes.GroupDetail{invGroup("g", map[string]string{"p": `not a policy`})}}},
+ "a managed policy that does not decode": {{Policies: []iamtypes.ManagedPolicyDetail{invManaged(managed, `not a policy`)}}},
+ "a user in a group the listing did not return": {{UserDetailList: []iamtypes.UserDetail{invUser("u", []string{"ghosts"}, nil)}}},
+ "a principal with no ARN": {{RoleDetailList: []iamtypes.RoleDetail{{RoleName: aws.String("r"), AssumeRolePolicyDocument: encoded(trustEC2)}}}},
+ } {
+ p, _ := inventoryFor(t, nil, pages...)
+ if inv, err := p.BucketPrincipals(ctx, bucket); err == nil || errors.Is(err, errInventoryRepeated) {
+ t.Errorf("%s: %+v %v; want an error of its own", name, inv, err)
+ }
+ }
+ // A name that is a pattern is not a name, and the account is not read
+ // for it.
+ for _, name := range []string{"", "carlos-store-*", "a/b", "arn:aws:s3:::b", "a?b", "a${x}"} {
+ p, stub := inventoryFor(t, nil, good())
+ if _, err := p.BucketPrincipals(ctx, name); err == nil || len(stub.calls) != 0 {
+ t.Errorf("the name %q: %v after %d calls", name, err, len(stub.calls))
+ }
+ }
+}
+
+type endlessInventory struct{ calls int }
+
+func (e *endlessInventory) GetAccountAuthorizationDetails(context.Context, *iam.GetAccountAuthorizationDetailsInput, ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error) {
+ e.calls++
+ if e.calls > 10 {
+ // Past the bound the test sets: a page that must never be asked
+ // for. It ends the listing, so a report without the bound would
+ // answer.
+ return &iam.GetAccountAuthorizationDetailsOutput{}, nil
+ }
+ return &iam.GetAccountAuthorizationDetailsOutput{IsTruncated: true, Marker: aws.String(fmt.Sprint("m", e.calls))}, nil
+}
+
+// A declared platform principal is one principal, spelled out.
+func TestPlatformPrincipalsAreDeclaredOneByOne(t *testing.T) {
+ for _, arn := range []string{
+ "arn:aws:iam::123456789012:role/carlos-flagship-host",
+ "arn:aws:iam::123456789012:user/ops/someone",
+ "arn:aws:iam::123456789012:group/admins",
+ "arn:aws-us-gov:iam::123456789012:role/x",
+ } {
+ if err := validPlatformPrincipal(arn); err != nil {
+ t.Errorf("%s: %v", arn, err)
+ }
+ }
+ for _, arn := range []string{
+ "", "*", "carlos-flagship-host",
+ "arn:aws:iam::123456789012:role/*",
+ "arn:aws:iam::123456789012:role/carlos-*",
+ "arn:aws:iam::*:role/carlos-flagship-host",
+ "arn:aws:iam::123456789012:root",
+ "arn:aws:iam::123456789012:policy/p",
+ "arn:aws:iam::123456789012:role/",
+ "arn:aws:sts::123456789012:assumed-role/r/s",
+ "arn:aws:iam::12345:role/r",
+ "arn:aws:iam:eu-west-1:123456789012:role/r",
+ "arn:aws:s3:::carlos-store-bac-blog",
+ "arn:aws:iam::123456789012:role/${aws:username}",
+ } {
+ if err := validPlatformPrincipal(arn); err == nil {
+ t.Errorf("%q was accepted as a platform principal", arn)
+ }
+ }
+ if _, err := NewPrincipalReport(&stubInventory{}, []string{"arn:aws:iam::123456789012:role/*"}); err == nil {
+ t.Error("a pattern was accepted by the constructor")
+ }
+ if _, err := NewPrincipalReport(nil, nil); err == nil {
+ t.Error("a report was built over no client")
+ }
+}
+
+// Nothing here is a GranteeProber: a listing of the account is a report
+// for a person, and the grant walk on AWS claims nothing, as before.
+func TestNothingOnAWSAnswersForGrantees(t *testing.T) {
+ var plain Provisioner = &awsProvisioner{}
+ if _, ok := plain.(GranteeProber); ok {
+ t.Fatal("the AWS provisioner answers for grantees")
+ }
+ if _, ok := plain.(BucketProber); !ok {
+ t.Fatal("the AWS provisioner lost its bucket probe")
+ }
+ report, err := NewPrincipalReport(&stubInventory{}, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if _, ok := any(report).(GranteeProber); ok {
+ t.Fatal("the principal report answers for grantees: one listing cannot support the ledger's claim")
+ }
+}
+
+// A page that is neither an answer nor an error (which only a stub
+// produces) is an error, not a panic.
+type nilPageInventory struct{}
+
+func (nilPageInventory) GetAccountAuthorizationDetails(context.Context, *iam.GetAccountAuthorizationDetailsInput, ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error) {
+ return nil, nil
+}
+
+func TestInventoryRefusesAPageThatIsNothing(t *testing.T) {
+ p, err := NewPrincipalReport(nilPageInventory{}, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if inv, err := p.BucketPrincipals(context.Background(), "carlos-store-bac-blog"); err == nil {
+ t.Fatalf("%+v with no error", inv)
+ }
+}
diff --git a/internal/objstore/provisioner.go b/internal/objstore/provisioner.go
index 59e1c3b29..4e26a00ba 100644
--- a/internal/objstore/provisioner.go
+++ b/internal/objstore/provisioner.go
@@ -47,8 +47,8 @@ type BucketProber interface {
// An absent bucket proves nothing about credentials: a key scoped to a
// bucket name outlives the bucket (a teardown that deleted the bucket and
// then failed to delete the user), and reaches a bucket of that name the
-// moment it is created again. Only an inventory of principals can show
-// that nobody is waiting on a name.
+// moment it is created again. No listing of an AWS account's
+// principals can show that nobody is waiting on a name either; see below.
//
// What an implementation must cover, or it must not exist: every kind of
// principal (users, roles and groups, not only the store users this package
@@ -61,10 +61,15 @@ type BucketProber interface {
// An implementation that cannot answer for all of these returns an error,
// and an error claims nothing.
//
-// The AWS provisioner does not implement it yet: it needs to list the store
-// users and read their policies, which the console's role is not granted
-// today. Until it does, no bucket's ownership ledger claims completeness on
-// AWS, and store transfer holds.
+// Nothing on AWS implements it. A report of who reaches a name exists
+// (principals.go) and is deliberately not this interface: one listing of
+// an eventually consistent account, of principals whose trust policies
+// let others act as them, about a name another actor can create first,
+// cannot support the claim. That needs a rule the account enforces and a
+// bucket only its own grant could have made
+// (docs/superpowers/plans/2026-10-02-aws-principal-inventory.md). Until
+// then no bucket's ownership ledger claims completeness on AWS, and store
+// transfer holds.
type GranteeProber interface {
BucketGrantees(ctx context.Context, name string) ([]string, error)
}