amadan

carlos / platform Public

The AWS principal inventory: who in the account has a policy that reaches a bucket name (built, unwired)

The grant walk may claim a bucket's ledger names everyone only when no
principal anywhere had a policy reaching the name (GranteeProber). The AWS
provisioner had no way to list principals, so on AWS no ledger claims
completeness and store transfer holds (OUTSTANDING 2).

principals.go lists the account once (iam:GetAccountAuthorizationDetails,
every page) and evaluates every identity policy itself against one bucket
name: users, roles and groups, inline and attached managed policies, a
user's groups, wildcards and policy variables, NotAction and NotResource.
It over-reports and never under-reports (a Deny, a condition or a boundary
is not evaluated), and what it cannot read is an error, which claims
nothing. It answers only for a name no bucket holds: a bucket's own
policy, ACL and access points are not read.

Inert: the plain AWS provisioner is unchanged and is not a GranteeProber;
the inventory is a separate constructor that nothing calls. Two things are
needed before it is wired, and neither is decided here
(docs/superpowers/plans/2026-10-02-aws-principal-inventory.md): which
principals are the platform's own (the host role and an administrator
role reach every store name, so with them counted no name is ever free),
and the IAM permission for the listing, which the host module withholds
today and this branch does not add.

Tests against a stub of the listing; twenty-six mutants in a tree copy.
Nothing was run against AWS.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
claude d61f4f2718c2c191e0b922536fa9251d0d426e7e parent 542dd6f
4 files changed, +1140 −4
  • docs/superpowers/plans/2026-10-02-aws-principal-inventory.md +54 −0
  • internal/objstore/principals.go +623 −0
  • internal/objstore/principals_test.go +456 −0
  • internal/objstore/provisioner.go +7 −4

CI passed — run details

diff --git a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md
new file mode 100644
index 000000000..a9ce70afb
--- /dev/null
+++ b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md
@@ -0,0 +1,54 @@
+# The AWS principal inventory
+
+Date: 2026-10-02. Status: built on `feat/carloku-principal-inventory-20261002`, **unwired**: no console constructs it, and nothing in production behaves differently. It is the first step of `2026-10-01-bucket-ownership-attestation.md` and of the store-transfer meter (`OUTSTANDING.md` item 2). It needs one decision and one permission before it can be switched on, both below. Nothing was run against AWS.
+
+## What it is for
+
+A bucket's ownership ledger may claim it names everyone who ever had the bucket only when the grant that minted the bucket could show two things: no bucket held the name, and no principal anywhere had a policy reaching the name (`objstore.GranteeProber`). The second needs a list of principals. The AWS provisioner had none, so on AWS no ledger claims completeness, no charged store transfer can be attributed, and the month holds.
+
+## What is built
+
+`internal/objstore/principals.go`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every policy document decoded and evaluated here against one bucket name.
+
+- **Covers:** users, roles and groups; inline and attached managed policies (the default version); a user's groups; resource patterns with `*`, `?` and policy variables; the bucket ARN and any object ARN under it; `NotAction` and `NotResource`.
+- **Over-reports, never under-reports.** A Deny, a condition, a permissions boundary or an organization policy only narrows access, and none is evaluated: a principal they keep out is still listed. A listed principal costs a claim. An omitted one would bill a tenant for bytes somebody else could move.
+- **Errors rather than guesses.** A document it cannot decode, an effect or statement shape it does not know, a managed policy attached and missing from the listing, a truncated or endless listing, a refused listing. An error claims nothing.
+- **Answers only for a name no bucket holds.** A bucket's own policy, its ACL, and access points exist only while the bucket does, and none is read. For a name a bucket holds the inventory returns an error.
+- **Inert by default.** The plain AWS provisioner is unchanged and is not a `GranteeProber`. The inventory is a separate constructor (`NewAWSProvisionerWithInventory`) that nothing calls.
+
+Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestThePlainAWS'`, against a stub of the IAM listing. No test reads a real account.
+
+Mutation checks, in a tree copy, twenty-six: `NotResource` and `NotAction` not reaching; actions compared with case; a policy variable taken literally; object ARNs, and the bucket ARN, not matched; the empty key counted; a missing managed policy skipped; group membership ignored; the platform counted among grantees; a name a bucket holds answered for; only the first page read; any version of a managed policy taken as the default; an unknown effect passed over quietly; a pattern accepted as a platform principal; a question mark taken literally; attached policies not read; the listing filtered; roles not evaluated; a reaching group not listed; an Allow with no resource passed over quietly; the partition fixed; a failed bucket probe read as no bucket; a service pattern compared as text; a plus sign read as a space; the page limit not enforced. One survived the first run (the partition was never derived in a test) and has a test now. The page-limit mutant dies by the test never ending, not by an assertion.
+
+## What it does not do
+
+- **Existing and adopted buckets.** The four stores already on the flagship, and any adopted bucket, need the resource side (bucket policy, ACL, access points, grants to other accounts) and the attestation door. Not built.
+- **Anything after the instant of the listing.** IAM is eventually consistent, and a policy attached by hand later is not known to the ledger. The claim is dated. Nothing re-checks it. Whether a periodic re-check should exist, and what a new grantee found later does to months already billed, is not designed.
+- **The root user.** No policy lists it.
+- **A real listing.** The shape of a real account's documents (encodings, service-linked roles, Identity Center roles, AWS managed policies with constructs this parser refuses) is unverified. The first real run may error, which holds as today.
+
+## Decision for Paul: which principals are the platform's own
+
+In a real account some principals reach every store bucket name. The host role provisions them (`CreateBucket`, `DeleteBucket`, `ListBucket`, `DeleteObject` on the store prefix; no `GetObject`). `OrganizationAccountAccessRole` and any administrator role reach everything. Counted as grantees, they make every name "already granted", and no ledger ever claims completeness.
+
+The inventory therefore takes a list of principals declared the platform's own, by exact ARN (no patterns). They are reported apart and are not grantees. **The list is empty and nothing sets it.**
+
+The question is not technical. AWS charges transfer by bucket and the report does not say who moved the bytes. If an administrator downloads from a tenant's bucket, that transfer is on the tenant's bucket and would be billed to the tenant.
+
+Proposed, for review:
+
+1. Declared by ARN in the deployment's configuration, and written into each ledger's evidence so a statement can show what the claim rested on.
+2. Only two kinds may be declared: the deployment's own host role, and roles only a person with account administration can assume.
+3. Staff do not read tenant store buckets. If support ever requires it, the bytes are small and the platform absorbs them by credit, case by case.
+
+Without an answer the inventory stays unwired.
+
+## Approval needed: one IAM permission
+
+`iam:GetAccountAuthorizationDetails` on the console's role, account 271376211898. It reads every policy document in the account. The host module today deliberately withholds policy reads from that role (`infra/modules/region-host/main.tf`, the inventory page's grant). This is a wider read than anything the console has. The module is **not changed** on this branch.
+
+Alternative: run the listing from an operator's own credentials at attestation time, through a verb, and hand the console the result. The console's role gains nothing, at the cost of a person in the loop for every new store. Not built.
+
+## After both
+
+Wire the constructor in `cmd/console` behind the declaration, run it read-only against the flagship, and compare its answer for one new name with the account by hand. Then the resource side and the attestation door for existing buckets.
diff --git a/internal/objstore/principals.go b/internal/objstore/principals.go
new file mode 100644
index 000000000..b7ebbd6af
--- /dev/null
+++ b/internal/objstore/principals.go
@@ -0,0 +1,623 @@
+package objstore
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "net/url"
+ "sort"
+ "strings"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ "github.com/aws/aws-sdk-go-v2/service/iam"
+)
+
+// The AWS principal inventory: who in the account has a policy that
+// reaches a bucket name (GranteeProber, provisioner.go).
+//
+// It answers one question, for a name NO BUCKET HOLDS: which users, roles
+// and groups of this account carry an identity policy, inline or attached,
+// directly or through a group, with an Allow statement whose resource
+// matches the bucket's ARN or an object ARN under it and whose action can
+// be an S3 action. It reads the whole account's authorization details in
+// one listing (iam:GetAccountAuthorizationDetails) and evaluates every
+// document itself.
+//
+// It over-reports and never under-reports, because a principal wrongly
+// listed costs a claim (the ledger claims nothing, the meter holds) and a
+// principal wrongly omitted bills a tenant for bytes somebody else could
+// move:
+//
+// - Deny statements, conditions, permissions boundaries, session
+// policies and organization policies only narrow access. None is
+// evaluated. A principal an explicit Deny keeps out is still listed.
+// - NotAction and NotResource on an Allow are taken to reach the bucket.
+// - A policy variable (${aws:username}) is taken to expand to anything.
+// - A group whose policy reaches the bucket is listed, with or without
+// members, and so is each of its members.
+//
+// What it cannot answer is an error, and an error claims nothing:
+//
+// - a document it cannot decode, or a statement that is not the IAM
+// grammar it knows (an Effect that is neither Allow nor Deny; an Allow
+// with no resource element);
+// - an attached managed policy the listing did not return;
+// - a listing that is truncated without a marker, or does not end;
+// - a name a bucket holds. Resource-side grants (a bucket policy, an
+// ACL, an access point and its policy, a grant to another account)
+// exist only while a bucket does, and none of them is read here. For
+// an existing bucket this inventory is half an answer, so it gives
+// none. The attestation of buckets that already exist needs that
+// other half; it is not built.
+//
+// What it does not see, by construction, and what follows from that:
+//
+// - The account's root user, which no policy lists. It is the
+// platform's.
+// - Anything that changes after the listing. The answer is true of one
+// instant; IAM is also eventually consistent, so a policy attached
+// seconds before may be missing. A ledger's claim built on it is
+// dated, and a principal given the name later, outside the platform's
+// own grant, is not known to the ledger.
+//
+// Platform principals. In any real account some principals reach every
+// store bucket name: the console's own role provisions them, and an
+// administrator role reaches everything. With those counted no name is
+// ever free of grantees. The deployment may therefore declare principals
+// that are its own, by exact ARN; they are reported apart and are not
+// grantees. Which principals may be declared, and whether the bytes they
+// move are the tenant's to pay for, is a decision that has not been made
+// (docs/superpowers/plans/2026-10-02-aws-principal-inventory.md): nothing
+// in the console declares any, and nothing in the console constructs this
+// inventory. Until both are decided and the console's role may make the
+// listing, the AWS provisioner answers as it always has.
+
+// iamInventoryAPI is the one IAM read the inventory makes.
+type iamInventoryAPI interface {
+ GetAccountAuthorizationDetails(ctx context.Context, in *iam.GetAccountAuthorizationDetailsInput, optFns ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error)
+}
+
+// inventoryMaxPages bounds the listing: a listing that has not ended by
+// then is not trusted to end. At the service's page size of a hundred
+// entities this is far more than an account can hold.
+const inventoryMaxPages = 2000
+
+// ErrBucketExistsNoInventory is the inventory's answer for a name a bucket
+// holds: its resource-side grants are not read, so nothing is claimed.
+var ErrBucketExistsNoInventory = errors.New("objstore: the bucket exists, and its bucket policy, ACL and access points are not inventoried")
+
+// PrincipalGrant is one principal whose identity policies reach a bucket
+// name, and the policies through which they do.
+type PrincipalGrant struct {
+ ARN string `json:"arn"`
+ Kind string `json:"kind"` // user | role | group
+ // Via names each policy that reaches the bucket: "inline:<name>",
+ // "managed:<policy arn>", and for a user "group:<group arn>" for each
+ // of its groups that does. Sorted.
+ Via []string `json:"via"`
+}
+
+// PrincipalInventory is everything the listing showed about one name.
+type PrincipalInventory struct {
+ Bucket string `json:"bucket"`
+ // Grantees reach the name and are not declared the platform's own.
+ Grantees []PrincipalGrant `json:"grantees"`
+ // Platform reach the name and were declared the platform's own.
+ Platform []PrincipalGrant `json:"platform"`
+ // What was read, so an empty answer can be told from an empty account.
+ Users, Roles, Groups, ManagedPolicies int
+}
+
+// awsInventoryProvisioner is the AWS provisioner with the principal
+// inventory: the only AWS provisioner that is a GranteeProber.
+type awsInventoryProvisioner struct {
+ *awsProvisioner
+ inv iamInventoryAPI
+ platform map[string]bool
+}
+
+var (
+ _ Provisioner = (*awsInventoryProvisioner)(nil)
+ _ BucketProber = (*awsInventoryProvisioner)(nil)
+ _ GranteeProber = (*awsInventoryProvisioner)(nil)
+)
+
+// NewAWSProvisionerWithInventory builds the AWS provisioner with the
+// principal inventory. platform is the deployment's own principals, by
+// exact ARN (see the note at the top of this file: nothing decides yet
+// which may be declared, and no console calls this).
+func NewAWSProvisionerWithInventory(ctx context.Context, region, endpoint, boundary string, platform []string) (Provisioner, error) {
+ p, err := NewAWSProvisioner(ctx, region, endpoint, boundary)
+ if err != nil {
+ return nil, err
+ }
+ base := p.(*awsProvisioner)
+ inv, ok := base.iam.(iamInventoryAPI)
+ if !ok {
+ return nil, errors.New("objstore: the IAM client cannot list the account's authorization details")
+ }
+ return withInventory(base, inv, platform)
+}
+
+func withInventory(base *awsProvisioner, inv iamInventoryAPI, platform []string) (*awsInventoryProvisioner, error) {
+ declared := map[string]bool{}
+ for _, arn := range platform {
+ if err := validPlatformPrincipal(arn); err != nil {
+ return nil, err
+ }
+ declared[arn] = true
+ }
+ return &awsInventoryProvisioner{awsProvisioner: base, inv: inv, platform: declared}, nil
+}
+
+// validPlatformPrincipal accepts the ARN of one user, role or group,
+// spelled out: no wildcard, no pattern. A declaration is a list a person
+// can read, each entry one principal.
+func validPlatformPrincipal(arn string) error {
+ parts := strings.SplitN(arn, ":", 6)
+ if len(parts) != 6 || parts[0] != "arn" || parts[1] == "" || parts[2] != "iam" || parts[3] != "" || len(parts[4]) != 12 || strings.Trim(parts[4], "0123456789") != "" {
+ return fmt.Errorf("objstore: platform principal %q is not an IAM principal ARN", arn)
+ }
+ kind, name, ok := strings.Cut(parts[5], "/")
+ if !ok || name == "" || (kind != "user" && kind != "role" && kind != "group") {
+ return fmt.Errorf("objstore: platform principal %q is not a user, role or group", arn)
+ }
+ if strings.ContainsAny(arn, "*?$ \t\n") {
+ return fmt.Errorf("objstore: platform principal %q is a pattern; each principal is declared by its own ARN", arn)
+ }
+ return nil
+}
+
+// BucketGrantees lists the principals, other than the platform's declared
+// own, whose identity policies reach the name (GranteeProber). It answers
+// only for a name no bucket holds.
+func (p *awsInventoryProvisioner) BucketGrantees(ctx context.Context, name string) ([]string, error) {
+ inv, err := p.BucketPrincipals(ctx, name)
+ if err != nil {
+ return nil, err
+ }
+ out := make([]string, 0, len(inv.Grantees))
+ for _, g := range inv.Grantees {
+ out = append(out, g.ARN)
+ }
+ return out, nil
+}
+
+// BucketPrincipals is the inventory in full, for a caller that records
+// what it rested on.
+func (p *awsInventoryProvisioner) BucketPrincipals(ctx context.Context, name string) (*PrincipalInventory, error) {
+ if name == "" || strings.ContainsAny(name, "*?/:${} ") {
+ return nil, fmt.Errorf("objstore: %q is not a bucket name", name)
+ }
+ account, err := p.listAccount(ctx)
+ if err != nil {
+ return nil, err
+ }
+ inv, err := account.reaching(name, p.platform)
+ if err != nil {
+ return nil, err
+ }
+ // Last, so the window between this answer and the caller's use of it
+ // is as short as this function can make it: a name a bucket holds has
+ // grants this inventory did not read.
+ exists, err := p.BucketExists(ctx, name)
+ if err != nil {
+ return nil, err
+ }
+ if exists {
+ return nil, fmt.Errorf("%w: %s", ErrBucketExistsNoInventory, name)
+ }
+ return inv, nil
+}
+
+// accountPolicies is the account's authorization details, decoded.
+type accountPolicies struct {
+ users, roles, groups []inventoryPrincipal
+ managed map[string]*policyDoc // by policy ARN
+}
+
+type inventoryPrincipal struct {
+ arn, kind, name string
+ inline map[string]*policyDoc // by policy name
+ attached []string // managed policy ARNs
+ groups []string // group names, for a user
+}
+
+func (p *awsInventoryProvisioner) listAccount(ctx context.Context) (*accountPolicies, error) {
+ acct := &accountPolicies{managed: map[string]*policyDoc{}}
+ var marker *string
+ seen := map[string]bool{}
+ for page := 0; ; page++ {
+ if page >= inventoryMaxPages {
+ return nil, fmt.Errorf("objstore: the account's authorization details did not end within %d pages", inventoryMaxPages)
+ }
+ out, err := p.inv.GetAccountAuthorizationDetails(ctx, &iam.GetAccountAuthorizationDetailsInput{Marker: marker})
+ if err != nil {
+ return nil, fmt.Errorf("objstore: list the account's authorization details: %w", err)
+ }
+ for _, u := range out.UserDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(u.Arn), kind: "user", name: aws.ToString(u.UserName), groups: u.GroupList, inline: map[string]*policyDoc{}}
+ for _, d := range u.UserPolicyList {
+ doc, err := decodePolicy(aws.ToString(d.PolicyDocument))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: policy %q of %s: %w", aws.ToString(d.PolicyName), pr.arn, err)
+ }
+ pr.inline[aws.ToString(d.PolicyName)] = doc
+ }
+ for _, a := range u.AttachedManagedPolicies {
+ pr.attached = append(pr.attached, aws.ToString(a.PolicyArn))
+ }
+ acct.users = append(acct.users, pr)
+ }
+ for _, g := range out.GroupDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(g.Arn), kind: "group", name: aws.ToString(g.GroupName), inline: map[string]*policyDoc{}}
+ for _, d := range g.GroupPolicyList {
+ doc, err := decodePolicy(aws.ToString(d.PolicyDocument))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: policy %q of %s: %w", aws.ToString(d.PolicyName), pr.arn, err)
+ }
+ pr.inline[aws.ToString(d.PolicyName)] = doc
+ }
+ for _, a := range g.AttachedManagedPolicies {
+ pr.attached = append(pr.attached, aws.ToString(a.PolicyArn))
+ }
+ acct.groups = append(acct.groups, pr)
+ }
+ for _, r := range out.RoleDetailList {
+ pr := inventoryPrincipal{arn: aws.ToString(r.Arn), kind: "role", name: aws.ToString(r.RoleName), inline: map[string]*policyDoc{}}
+ for _, d := range r.RolePolicyList {
+ doc, err := decodePolicy(aws.ToString(d.PolicyDocument))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: policy %q of %s: %w", aws.ToString(d.PolicyName), pr.arn, err)
+ }
+ pr.inline[aws.ToString(d.PolicyName)] = doc
+ }
+ for _, a := range r.AttachedManagedPolicies {
+ pr.attached = append(pr.attached, aws.ToString(a.PolicyArn))
+ }
+ acct.roles = append(acct.roles, pr)
+ }
+ for _, m := range out.Policies {
+ arn := aws.ToString(m.Arn)
+ var current *string
+ for _, v := range m.PolicyVersionList {
+ if v.IsDefaultVersion {
+ if current != nil {
+ return nil, fmt.Errorf("objstore: managed policy %s has two default versions", arn)
+ }
+ current = v.Document
+ }
+ }
+ if current == nil {
+ return nil, fmt.Errorf("objstore: managed policy %s has no default version in the listing", arn)
+ }
+ doc, err := decodePolicy(aws.ToString(current))
+ if err != nil {
+ return nil, fmt.Errorf("objstore: managed policy %s: %w", arn, err)
+ }
+ acct.managed[arn] = doc
+ }
+ if !out.IsTruncated {
+ break
+ }
+ next := aws.ToString(out.Marker)
+ if next == "" || seen[next] {
+ return nil, errors.New("objstore: the account's authorization details are truncated with no way on")
+ }
+ seen[next] = true
+ marker = out.Marker
+ }
+ for _, list := range [][]inventoryPrincipal{acct.users, acct.roles, acct.groups} {
+ for _, pr := range list {
+ if pr.arn == "" || pr.name == "" {
+ return nil, errors.New("objstore: the account's authorization details name a principal with no ARN or name")
+ }
+ }
+ }
+ return acct, nil
+}
+
+// reaching evaluates every principal against one bucket name.
+func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*PrincipalInventory, error) {
+ inv := &PrincipalInventory{Bucket: bucket, Grantees: []PrincipalGrant{}, Platform: []PrincipalGrant{}, Users: len(a.users), Roles: len(a.roles), Groups: len(a.groups), ManagedPolicies: len(a.managed)}
+ via := func(pr inventoryPrincipal) ([]string, error) {
+ partition := "aws"
+ if parts := strings.SplitN(pr.arn, ":", 3); len(parts) == 3 && parts[0] == "arn" && parts[1] != "" {
+ partition = parts[1]
+ }
+ var out []string
+ for name, doc := range pr.inline {
+ if doc.reaches(partition, bucket) {
+ out = append(out, "inline:"+name)
+ }
+ }
+ for _, arn := range pr.attached {
+ doc, ok := a.managed[arn]
+ if !ok {
+ return nil, fmt.Errorf("objstore: %s has managed policy %s attached, and the listing did not return it", pr.arn, arn)
+ }
+ if doc.reaches(partition, bucket) {
+ out = append(out, "managed:"+arn)
+ }
+ }
+ return out, nil
+ }
+ groupVia := map[string][]string{}
+ groupARN := map[string]string{}
+ add := func(pr inventoryPrincipal, through []string) {
+ if len(through) == 0 {
+ return
+ }
+ sort.Strings(through)
+ g := PrincipalGrant{ARN: pr.arn, Kind: pr.kind, Via: through}
+ if platform[pr.arn] {
+ inv.Platform = append(inv.Platform, g)
+ } else {
+ inv.Grantees = append(inv.Grantees, g)
+ }
+ }
+ for _, g := range a.groups {
+ through, err := via(g)
+ if err != nil {
+ return nil, err
+ }
+ if _, twice := groupARN[g.name]; twice {
+ return nil, fmt.Errorf("objstore: the listing names the group %q twice", g.name)
+ }
+ groupVia[g.name], groupARN[g.name] = through, g.arn
+ add(g, through)
+ }
+ for _, u := range a.users {
+ through, err := via(u)
+ if err != nil {
+ return nil, err
+ }
+ for _, name := range u.groups {
+ arn, ok := groupARN[name]
+ if !ok {
+ return nil, fmt.Errorf("objstore: %s is in the group %q, and the listing did not return it", u.arn, name)
+ }
+ if len(groupVia[name]) > 0 {
+ through = append(through, "group:"+arn)
+ }
+ }
+ add(u, through)
+ }
+ for _, r := range a.roles {
+ through, err := via(r)
+ if err != nil {
+ return nil, err
+ }
+ add(r, through)
+ }
+ for _, list := range []*[]PrincipalGrant{&inv.Grantees, &inv.Platform} {
+ sort.Slice(*list, func(i, j int) bool { return (*list)[i].ARN < (*list)[j].ARN })
+ }
+ return inv, nil
+}
+
+// policyDoc is one identity policy, as much of it as the inventory reads.
+type policyDoc struct {
+ statements []policyStmt
+}
+
+type policyStmt struct {
+ allow bool
+ actions, notActions []string
+ resources, notResources []string
+ hasNotAction, hasNotResource bool
+}
+
+// oneOrMany decodes an IAM element that is a string or a list of strings.
+type oneOrMany []string
+
+func (o *oneOrMany) UnmarshalJSON(raw []byte) error {
+ var one string
+ if err := json.Unmarshal(raw, &one); err == nil {
+ *o = []string{one}
+ return nil
+ }
+ var many []string
+ if err := json.Unmarshal(raw, &many); err != nil {
+ return errors.New("neither a string nor a list of strings")
+ }
+ *o = many
+ return nil
+}
+
+// decodePolicy reads a policy document as IAM returns it: percent-encoded
+// JSON (RFC 3986; a plus sign is a plus sign), or plain JSON.
+func decodePolicy(raw string) (*policyDoc, error) {
+ text := strings.TrimSpace(raw)
+ if text == "" {
+ return nil, errors.New("an empty policy document")
+ }
+ if !strings.HasPrefix(text, "{") {
+ decoded, err := url.PathUnescape(text)
+ if err != nil {
+ return nil, fmt.Errorf("undecodable policy document: %w", err)
+ }
+ text = decoded
+ }
+ var doc struct {
+ Statement json.RawMessage `json:"Statement"`
+ }
+ if err := json.Unmarshal([]byte(text), &doc); err != nil {
+ return nil, fmt.Errorf("policy document is not JSON: %w", err)
+ }
+ if len(doc.Statement) == 0 {
+ return nil, errors.New("policy document has no Statement")
+ }
+ type stmt struct {
+ Effect string `json:"Effect"`
+ Action *oneOrMany `json:"Action"`
+ NotAction *oneOrMany `json:"NotAction"`
+ Resource *oneOrMany `json:"Resource"`
+ NotResource *oneOrMany `json:"NotResource"`
+ }
+ var list []stmt
+ if err := json.Unmarshal(doc.Statement, &list); err != nil {
+ var one stmt
+ if err := json.Unmarshal(doc.Statement, &one); err != nil {
+ return nil, fmt.Errorf("policy Statement is neither a statement nor a list of them: %w", err)
+ }
+ list = []stmt{one}
+ }
+ out := &policyDoc{}
+ for i, s := range list {
+ var ps policyStmt
+ switch s.Effect {
+ case "Allow":
+ ps.allow = true
+ case "Deny":
+ // Read no further: a Deny only narrows, and is not evaluated.
+ out.statements = append(out.statements, ps)
+ continue
+ default:
+ return nil, fmt.Errorf("statement %d has the effect %q", i, s.Effect)
+ }
+ if (s.Action == nil) == (s.NotAction == nil) {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Action and NotAction", i)
+ }
+ if (s.Resource == nil) == (s.NotResource == nil) {
+ return nil, fmt.Errorf("statement %d does not have exactly one of Resource and NotResource", i)
+ }
+ if s.Action != nil {
+ ps.actions = *s.Action
+ } else {
+ ps.hasNotAction = true
+ }
+ if s.Resource != nil {
+ ps.resources = *s.Resource
+ } else {
+ ps.hasNotResource = true
+ }
+ out.statements = append(out.statements, ps)
+ }
+ return out, nil
+}
+
+// reaches reports whether any Allow statement of the document can apply
+// an S3 action to the bucket or to an object under it.
+func (d *policyDoc) reaches(partition, bucket string) bool {
+ arn := "arn:" + partition + ":s3:::" + bucket
+ for _, s := range d.statements {
+ if !s.allow {
+ continue
+ }
+ action := s.hasNotAction
+ for _, a := range s.actions {
+ if actionCanBeS3(a) {
+ action = true
+ }
+ }
+ if !action {
+ continue
+ }
+ if s.hasNotResource {
+ return true
+ }
+ for _, r := range s.resources {
+ if resourceReaches(r, arn) {
+ return true
+ }
+ }
+ }
+ return false
+}
+
+// actionCanBeS3 reports whether an action pattern can name an action of
+// the S3 service. Action names are not case-sensitive.
+func actionCanBeS3(pattern string) bool {
+ p := strings.ToLower(conservativePattern(pattern))
+ service, _, ok := strings.Cut(p, ":")
+ if !ok {
+ // No service prefix: "*", or something that is not an action; a
+ // pattern that can match "s3:" followed by anything is taken to.
+ return wildcardLive(p, "s3:") != nil
+ }
+ return wildcardMatch(service, "s3")
+}
+
+// resourceReaches reports whether a resource pattern matches the bucket's
+// ARN, or can match the ARN of some object under it.
+func resourceReaches(pattern, bucketARN string) bool {
+ p := conservativePattern(pattern)
+ if wildcardMatch(p, bucketARN) {
+ return true
+ }
+ // An object ARN is the bucket's, a slash, and a key of at least one
+ // character. The pattern matches one if, having matched "<arn>/", some
+ // of it is left: what is left of a pattern always matches some string
+ // of at least one character (a star can stand for one). A pattern
+ // used up exactly at the slash matches only the empty key, which no
+ // object has.
+ for at := range wildcardLive(p, bucketARN+"/") {
+ if at < len([]rune(p)) {
+ return true
+ }
+ }
+ return false
+}
+
+// conservativePattern replaces each policy variable with a star: a
+// variable expands to a value this inventory does not know.
+func conservativePattern(pattern string) string {
+ for {
+ i := strings.Index(pattern, "${")
+ if i < 0 {
+ return pattern
+ }
+ j := strings.Index(pattern[i:], "}")
+ if j < 0 {
+ return pattern[:i] + "*"
+ }
+ pattern = pattern[:i] + "*" + pattern[i+j+1:]
+ }
+}
+
+// wildcardLive runs an IAM wildcard pattern (* any run, ? any one
+// character) over text and returns the set of pattern positions it can be
+// at once all of text is consumed, or nil if it cannot consume it.
+func wildcardLive(pattern, text string) map[int]bool {
+ p := []rune(pattern)
+ closure := func(set map[int]bool) {
+ for changed := true; changed; {
+ changed = false
+ for i := range set {
+ if i < len(p) && p[i] == '*' && !set[i+1] {
+ set[i+1] = true
+ changed = true
+ }
+ }
+ }
+ }
+ live := map[int]bool{0: true}
+ closure(live)
+ for _, c := range text {
+ next := map[int]bool{}
+ for i := range live {
+ if i >= len(p) {
+ continue
+ }
+ switch {
+ case p[i] == '*':
+ next[i] = true
+ case p[i] == '?' || p[i] == c:
+ next[i+1] = true
+ }
+ }
+ if len(next) == 0 {
+ return nil
+ }
+ closure(next)
+ live = next
+ }
+ return live
+}
+
+// wildcardMatch reports whether the pattern matches the whole of text.
+func wildcardMatch(pattern, text string) bool {
+ return wildcardLive(pattern, text)[len([]rune(pattern))]
+}
diff --git a/internal/objstore/principals_test.go b/internal/objstore/principals_test.go
new file mode 100644
index 000000000..151321b71
--- /dev/null
+++ b/internal/objstore/principals_test.go
@@ -0,0 +1,456 @@
+package objstore
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "net/url"
+ "reflect"
+ "strings"
+ "testing"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ "github.com/aws/aws-sdk-go-v2/service/iam"
+ iamtypes "github.com/aws/aws-sdk-go-v2/service/iam/types"
+ awss3 "github.com/aws/aws-sdk-go-v2/service/s3"
+)
+
+// stubInventory answers GetAccountAuthorizationDetails with fixed pages.
+type stubInventory struct {
+ pages []*iam.GetAccountAuthorizationDetailsOutput
+ err error
+ calls []string // the marker each call carried
+}
+
+func (s *stubInventory) GetAccountAuthorizationDetails(_ context.Context, in *iam.GetAccountAuthorizationDetailsInput, _ ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error) {
+ s.calls = append(s.calls, aws.ToString(in.Marker))
+ if s.err != nil {
+ return nil, s.err
+ }
+ if len(in.Filter) != 0 {
+ return nil, errors.New("the inventory filtered the listing: it must read every kind of entity")
+ }
+ i := len(s.calls) - 1
+ if i >= len(s.pages) {
+ return nil, fmt.Errorf("page %d asked for; there are %d", i, len(s.pages))
+ }
+ return s.pages[i], nil
+}
+
+const invAccount = "arn:aws:iam::123456789012:"
+
+// encoded is a policy document as IAM returns it: percent-encoded.
+func encoded(doc string) *string { return aws.String(url.PathEscape(doc)) }
+
+func allow(action, resource string) string {
+ return fmt.Sprintf(`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":%s,"Resource":%s}]}`, action, resource)
+}
+
+func invUser(name string, groups []string, inline map[string]string, attached ...string) iamtypes.UserDetail {
+ u := iamtypes.UserDetail{Arn: aws.String(invAccount + "user/" + name), UserName: aws.String(name), GroupList: groups}
+ for n, d := range inline {
+ u.UserPolicyList = append(u.UserPolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ u.AttachedManagedPolicies = append(u.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return u
+}
+
+func invRole(name string, inline map[string]string, attached ...string) iamtypes.RoleDetail {
+ r := iamtypes.RoleDetail{Arn: aws.String(invAccount + "role/" + name), RoleName: aws.String(name)}
+ for n, d := range inline {
+ r.RolePolicyList = append(r.RolePolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ r.AttachedManagedPolicies = append(r.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return r
+}
+
+func invGroup(name string, inline map[string]string, attached ...string) iamtypes.GroupDetail {
+ g := iamtypes.GroupDetail{Arn: aws.String(invAccount + "group/" + name), GroupName: aws.String(name)}
+ for n, d := range inline {
+ g.GroupPolicyList = append(g.GroupPolicyList, iamtypes.PolicyDetail{PolicyName: aws.String(n), PolicyDocument: encoded(d)})
+ }
+ for _, a := range attached {
+ g.AttachedManagedPolicies = append(g.AttachedManagedPolicies, iamtypes.AttachedPolicy{PolicyArn: aws.String(a)})
+ }
+ return g
+}
+
+func invManaged(arn, doc string) iamtypes.ManagedPolicyDetail {
+ return iamtypes.ManagedPolicyDetail{Arn: aws.String(arn), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), Document: encoded(allow(`"s3:*"`, `"*"`))}, // an old version that reaches everything: not the default, not read
+ {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(doc)},
+ }}
+}
+
+// inventoryFor builds the provisioner over fixed pages and a bucket that
+// does or does not exist.
+func inventoryFor(t *testing.T, exists bool, platform []string, pages ...*iam.GetAccountAuthorizationDetailsOutput) (*awsInventoryProvisioner, *stubInventory) {
+ t.Helper()
+ s3 := &stubS3{t: t, listObjectsV2: func(*awss3.ListObjectsV2Input) (*awss3.ListObjectsV2Output, error) {
+ if exists {
+ return &awss3.ListObjectsV2Output{}, nil
+ }
+ return nil, &smithyAPIErrorStub{code: "NoSuchBucket"}
+ }}
+ inv := &stubInventory{pages: pages}
+ p, err := withInventory(&awsProvisioner{s3: s3, iam: &stubIAM{t: t}, region: "eu-west-1"}, inv, platform)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return p, inv
+}
+
+func arns(list []PrincipalGrant) []string {
+ out := []string{}
+ for _, g := range list {
+ out = append(out, g.ARN)
+ }
+ return out
+}
+
+// The matcher: which resource and action patterns reach a bucket name.
+func TestPolicyPatternsThatReachABucket(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ for _, tc := range []struct {
+ name, action, resource string
+ want bool
+ }{
+ {"the store user's own bucket statement", `["s3:ListBucket"]`, `"arn:aws:s3:::carlos-store-bac-blog"`, true},
+ {"the store user's own object statement", `["s3:GetObject","s3:PutObject"]`, `"arn:aws:s3:::carlos-store-bac-blog/*"`, true},
+ {"a prefix grant on a platform role", `"s3:CreateBucket"`, `"arn:aws:s3:::carlos-store-*"`, true},
+ {"a prefix grant on objects", `"s3:DeleteObject"`, `"arn:aws:s3:::carlos-*/*"`, true},
+ {"everything", `"*"`, `"*"`, true},
+ {"all of s3 on everything", `"s3:*"`, `"*"`, true},
+ {"a partial action pattern", `"s3:Get*"`, `"arn:aws:s3:::*"`, true},
+ {"an action in another case", `"S3:GETOBJECT"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`, true},
+ {"a question mark standing for one character", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blo?/*"`, true},
+ {"one object key", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/exports/2026.csv"`, true},
+ {"a policy variable", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-${aws:username}/*"`, true},
+ {"a policy variable left open", `"s3:GetObject"`, `"arn:aws:s3:::carlos-store-${aws:username"`, true},
+ {"any partition, region and account fields", `"s3:GetObject"`, `"arn:*:s3:*:*:carlos-store-bac-blog/*"`, true},
+ {"one of several resources", `"s3:GetObject"`, `["arn:aws:s3:::other","arn:aws:s3:::carlos-store-bac-blog/*"]`, true},
+ {"one of several actions", `["ec2:RunInstances","s3:ListBucket"]`, `"arn:aws:s3:::carlos-store-bac-blog"`, true},
+ {"a wildcard service", `"s*:List*"`, `"*"`, true},
+
+ {"another bucket", `"s3:*"`, `"arn:aws:s3:::carlos-store-bad-blog"`, false},
+ {"a bucket whose name this one begins", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac"`, false},
+ {"the objects of a bucket whose name this one begins", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac/*"`, false},
+ {"a bucket whose name begins with this one", `"s3:*"`, `["arn:aws:s3:::carlos-store-bac-blog2","arn:aws:s3:::carlos-store-bac-blog2/*"]`, false},
+ {"a question mark that needs one more character", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog?"`, false},
+ {"the slash and nothing after it", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog/"`, false},
+ {"a star after the name, which a slash and a key satisfy", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog*"`, true},
+ {"a question mark for the slash, then a star", `"s3:*"`, `"arn:aws:s3:::carlos-store-bac-blog?*"`, true},
+ {"another service on everything", `"ec2:*"`, `"*"`, false},
+ {"another service whose name begins s3", `"s3-object-lambda:*"`, `"*"`, false},
+ {"another partition", `"s3:*"`, `"arn:aws-cn:s3:::carlos-store-bac-blog"`, false},
+ {"no actions", `[]`, `"*"`, false},
+ {"no resources", `"s3:*"`, `[]`, false},
+ } {
+ doc, err := decodePolicy(allow(tc.action, tc.resource))
+ if err != nil {
+ t.Errorf("%s: %v", tc.name, err)
+ continue
+ }
+ if got := doc.reaches("aws", bucket); got != tc.want {
+ t.Errorf("%s: reaches %v, want %v", tc.name, got, tc.want)
+ }
+ }
+ for name, tc := range map[string]struct {
+ doc string
+ want bool
+ }{
+ "NotResource on an Allow": {`{"Statement":[{"Effect":"Allow","Action":"s3:GetObject","NotResource":"arn:aws:s3:::carlos-store-bac-blog/*"}]}`, true},
+ "NotAction on an Allow": {`{"Statement":[{"Effect":"Allow","NotAction":"s3:*","Resource":"*"}]}`, true},
+ "NotAction with another bucket": {`{"Statement":[{"Effect":"Allow","NotAction":"iam:*","Resource":"arn:aws:s3:::other"}]}`, false},
+ "a Deny alone": {`{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, false},
+ "an Allow that a Deny takes back": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"},{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, true},
+ "an Allow under a condition": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"Bool":{"aws:MultiFactorAuthPresent":"true"}}}]}`, true},
+ "one statement, not a list": {`{"Statement":{"Effect":"Allow","Action":"s3:*","Resource":"*"}}`, true},
+ "the second of two statements": {`{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"},{"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::carlos-store-bac-blog"}]}`, true},
+ } {
+ doc, err := decodePolicy(tc.doc)
+ if err != nil {
+ t.Errorf("%s: %v", name, err)
+ continue
+ }
+ if got := doc.reaches("aws", bucket); got != tc.want {
+ t.Errorf("%s: reaches %v, want %v", name, got, tc.want)
+ }
+ }
+ // What it cannot read is an error, never "does not reach".
+ for name, doc := range map[string]string{
+ "an unknown effect": `{"Statement":[{"Effect":"Permit","Action":"s3:*","Resource":"*"}]}`,
+ "an effect in the wrong case": `{"Statement":[{"Effect":"allow","Action":"s3:*","Resource":"*"}]}`,
+ "an Allow with no resource": `{"Statement":[{"Effect":"Allow","Action":"s3:*"}]}`,
+ "an Allow with no action": `{"Statement":[{"Effect":"Allow","Resource":"*"}]}`,
+ "both Resource and NotResource": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","NotResource":"*"}]}`,
+ "a resource that is a number": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":7}]}`,
+ "no statement": `{"Version":"2012-10-17"}`,
+ "not JSON": `Statement: none`,
+ "an empty document": ``,
+ "a broken percent-encoding": `%7B%2`,
+ "a statement that is a string": `{"Statement":"Allow"}`,
+ } {
+ if _, err := decodePolicy(doc); err == nil {
+ t.Errorf("%s: decoded; want an error", name)
+ }
+ }
+ // A plus sign in an encoded document is a plus sign, not a space.
+ doc, err := decodePolicy(`%7B%22Statement%22%3A%5B%7B%22Effect%22%3A%22Allow%22%2C%22Action%22%3A%22s3%3A%2A%22%2C%22Resource%22%3A%22arn%3Aaws%3As3%3A%3A%3Aa+b%22%7D%5D%7D`)
+ if err != nil || !doc.reaches("aws", "a+b") || doc.reaches("aws", "a b") {
+ t.Errorf("an encoded plus sign: %v", err)
+ }
+}
+
+// The inventory over an account: every kind of principal, inline and
+// managed, direct and through a group; the platform's declared own apart.
+func TestInventoryListsEveryPrincipalThatReachesTheName(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ const storeARN = `"arn:aws:s3:::carlos-store-bac-blog"`
+ const managedRead = "arn:aws:iam::123456789012:policy/store-readers"
+ const managedOther = "arn:aws:iam::123456789012:policy/ec2-only"
+ const awsAdmin = "arn:aws:iam::aws:policy/AdministratorAccess"
+ page1 := &iam.GetAccountAuthorizationDetailsOutput{
+ IsTruncated: true, Marker: aws.String("m1"),
+ UserDetailList: []iamtypes.UserDetail{
+ invUser("carlos-store-bac-blog", nil, map[string]string{"carlos-store": allow(`"s3:ListBucket"`, storeARN)}),
+ invUser("carlos-store-bad-shop", nil, map[string]string{"carlos-store": allow(`"s3:ListBucket"`, `"arn:aws:s3:::carlos-store-bad-shop"`)}),
+ invUser("in-a-reading-group", []string{"readers"}, nil),
+ invUser("in-an-idle-group", []string{"idle"}, nil),
+ invUser("with-a-managed-policy", nil, nil, managedRead),
+ invUser("with-another-managed-policy", nil, nil, managedOther),
+ },
+ GroupDetailList: []iamtypes.GroupDetail{
+ invGroup("readers", nil, managedRead),
+ invGroup("idle", map[string]string{"ec2": allow(`"ec2:*"`, `"*"`)}),
+ },
+ }
+ page2 := &iam.GetAccountAuthorizationDetailsOutput{
+ RoleDetailList: []iamtypes.RoleDetail{
+ invRole("carlos-flagship-host", map[string]string{"carlos-flagship-store": allow(`["s3:CreateBucket","s3:DeleteBucket"]`, `"arn:aws:s3:::carlos-store-*"`)}),
+ invRole("OrganizationAccountAccessRole", nil, awsAdmin),
+ invRole("lambda-thumbnails", map[string]string{"own": allow(`"s3:*"`, `["arn:aws:s3:::thumbnails","arn:aws:s3:::thumbnails/*"]`)}),
+ },
+ Policies: []iamtypes.ManagedPolicyDetail{
+ invManaged(managedRead, allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-*/*"`)),
+ invManaged(managedOther, allow(`"ec2:*"`, `"*"`)),
+ invManaged(awsAdmin, allow(`"*"`, `"*"`)),
+ },
+ }
+ p, stub := inventoryFor(t, false, nil, page1, page2)
+ inv, err := p.BucketPrincipals(context.Background(), bucket)
+ if err != nil {
+ t.Fatal(err)
+ }
+ want := []string{
+ invAccount + "group/readers",
+ invAccount + "role/OrganizationAccountAccessRole",
+ invAccount + "role/carlos-flagship-host",
+ invAccount + "user/carlos-store-bac-blog",
+ invAccount + "user/in-a-reading-group",
+ invAccount + "user/with-a-managed-policy",
+ }
+ if got := arns(inv.Grantees); !reflect.DeepEqual(got, want) {
+ t.Fatalf("grantees\n %v\nwant\n %v", got, want)
+ }
+ if len(inv.Platform) != 0 || inv.Users != 6 || inv.Groups != 2 || inv.Roles != 3 || inv.ManagedPolicies != 3 {
+ t.Fatalf("inventory %+v", inv)
+ }
+ if !reflect.DeepEqual(stub.calls, []string{"", "m1"}) {
+ t.Fatalf("pages asked for: %q; want the first, then the marker", stub.calls)
+ }
+ via := map[string][]string{}
+ for _, g := range inv.Grantees {
+ via[g.ARN] = g.Via
+ if wantKind := strings.SplitN(strings.TrimPrefix(g.ARN, invAccount), "/", 2)[0]; g.Kind != wantKind {
+ t.Errorf("%s has kind %q", g.ARN, g.Kind)
+ }
+ }
+ for arn, want := range map[string][]string{
+ invAccount + "user/carlos-store-bac-blog": {"inline:carlos-store"},
+ invAccount + "user/in-a-reading-group": {"group:" + invAccount + "group/readers"},
+ invAccount + "user/with-a-managed-policy": {"managed:" + managedRead},
+ invAccount + "group/readers": {"managed:" + managedRead},
+ invAccount + "role/carlos-flagship-host": {"inline:carlos-flagship-store"},
+ invAccount + "role/OrganizationAccountAccessRole": {"managed:" + awsAdmin},
+ } {
+ if !reflect.DeepEqual(via[arn], want) {
+ t.Errorf("%s reaches through %v; want %v", arn, via[arn], want)
+ }
+ }
+ // The GranteeProber's answer is the same list.
+ p, _ = inventoryFor(t, false, nil, page1, page2)
+ grantees, err := p.BucketGrantees(context.Background(), bucket)
+ if err != nil || !reflect.DeepEqual(grantees, want) {
+ t.Fatalf("BucketGrantees %v %v", grantees, err)
+ }
+ // With the platform's own declared, they are reported apart, and a
+ // name only they reach has no grantees.
+ platform := []string{invAccount + "role/carlos-flagship-host", invAccount + "role/OrganizationAccountAccessRole", invAccount + "role/not-in-the-account"}
+ p, _ = inventoryFor(t, false, platform, page1, page2)
+ inv, err = p.BucketPrincipals(context.Background(), bucket)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if got := arns(inv.Platform); !reflect.DeepEqual(got, []string{platform[1], platform[0]}) {
+ t.Fatalf("platform %v", got)
+ }
+ if got := arns(inv.Grantees); !reflect.DeepEqual(got, []string{want[0], want[3], want[4], want[5]}) {
+ t.Fatalf("grantees with the platform declared: %v", got)
+ }
+ p, _ = inventoryFor(t, false, platform, page1, page2)
+ if grantees, err := p.BucketGrantees(context.Background(), "carlos-store-new-name"); err != nil || !reflect.DeepEqual(grantees, []string{invAccount + "group/readers", invAccount + "user/in-a-reading-group", invAccount + "user/with-a-managed-policy"}) {
+ t.Fatalf("a new name under the prefix: %v %v; want the prefix readers, who are nobody's declared own", grantees, err)
+ }
+ p, _ = inventoryFor(t, false, platform, page1, page2)
+ if grantees, err := p.BucketGrantees(context.Background(), "elsewhere-entirely"); err != nil || grantees == nil || len(grantees) != 0 {
+ t.Fatalf("a name only the administrator reaches: %v %v; want none, and a list", grantees, err)
+ }
+}
+
+// A principal's policies are matched in the principal's own partition: a
+// bucket ARN is spelled with it.
+func TestInventoryMatchesInThePrincipalsPartition(t *testing.T) {
+ const bucket = "carlos-store-bac-blog"
+ cn := iamtypes.RoleDetail{Arn: aws.String("arn:aws-cn:iam::123456789012:role/reader"), RoleName: aws.String("reader"),
+ RolePolicyList: []iamtypes.PolicyDetail{{PolicyName: aws.String("p"), PolicyDocument: encoded(allow(`"s3:GetObject"`, `"arn:aws-cn:s3:::carlos-store-bac-blog/*"`))}}}
+ other := invRole("spelled-for-another-partition", map[string]string{"p": allow(`"s3:GetObject"`, `"arn:aws-cn:s3:::carlos-store-bac-blog/*"`)})
+ p, _ := inventoryFor(t, false, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{cn, other}})
+ grantees, err := p.BucketGrantees(context.Background(), bucket)
+ if err != nil || !reflect.DeepEqual(grantees, []string{"arn:aws-cn:iam::123456789012:role/reader"}) {
+ t.Fatalf("grantees %v %v; want the role whose partition its policy is spelled in, and not the one it is not", grantees, err)
+ }
+}
+
+// What the inventory cannot answer is an error: it never answers "nobody"
+// from a listing it could not read to the end.
+func TestInventoryClaimsNothingItCannotRead(t *testing.T) {
+ ctx := context.Background()
+ const bucket = "carlos-store-bac-blog"
+ good := &iam.GetAccountAuthorizationDetailsOutput{UserDetailList: []iamtypes.UserDetail{invUser("someone", nil, map[string]string{"p": allow(`"ec2:*"`, `"*"`)})}}
+
+ // The control: the same account, read to the end, reaches nothing.
+ p, _ := inventoryFor(t, false, nil, good)
+ if grantees, err := p.BucketGrantees(ctx, bucket); err != nil || len(grantees) != 0 {
+ t.Fatalf("control: %v %v", grantees, err)
+ }
+ // A name a bucket holds: its resource-side grants are not read.
+ p, _ = inventoryFor(t, true, nil, good)
+ if _, err := p.BucketGrantees(ctx, bucket); !errors.Is(err, ErrBucketExistsNoInventory) {
+ t.Fatalf("a name a bucket holds: %v; want ErrBucketExistsNoInventory", err)
+ }
+ // The probe failing is an error too.
+ p, _ = inventoryFor(t, false, nil, good)
+ p.s3 = &stubS3{t: t, listObjectsV2: func(*awss3.ListObjectsV2Input) (*awss3.ListObjectsV2Output, error) {
+ return nil, errors.New("timeout")
+ }}
+ if _, err := p.BucketGrantees(ctx, bucket); err == nil {
+ t.Fatal("a failed bucket probe: no error")
+ }
+ // The listing refused (the role may not make it).
+ p, stub := inventoryFor(t, false, nil, good)
+ stub.err = &smithyAPIErrorStub{code: "AccessDenied"}
+ if grantees, err := p.BucketGrantees(ctx, bucket); err == nil || grantees != nil {
+ t.Fatalf("a refused listing: %v %v", grantees, err)
+ }
+ for name, pages := range map[string][]*iam.GetAccountAuthorizationDetailsOutput{
+ "truncated with no marker": {{IsTruncated: true}},
+ "a marker that repeats": {{IsTruncated: true, Marker: aws.String("m")}, {IsTruncated: true, Marker: aws.String("m")}, {IsTruncated: true, Marker: aws.String("m")}},
+ "a managed policy attached and not listed": {{RoleDetailList: []iamtypes.RoleDetail{invRole("r", nil, "arn:aws:iam::123456789012:policy/missing")}}},
+ "a managed policy with no default version": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String("arn:aws:iam::123456789012:policy/p"), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), Document: encoded(allow(`"ec2:*"`, `"*"`))}}}}}},
+ "a managed policy with two default versions": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String("arn:aws:iam::123456789012:policy/p"), PolicyVersionList: []iamtypes.PolicyVersion{
+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(allow(`"ec2:*"`, `"*"`))}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(allow(`"ec2:*"`, `"*"`))}}}}}},
+ "an inline policy that does not decode": {{UserDetailList: []iamtypes.UserDetail{invUser("u", nil, map[string]string{"p": `{"Statement":[{"Effect":"Allow","Action":"s3:*"}]}`})}}},
+ "a role's inline policy that does not decode": {{RoleDetailList: []iamtypes.RoleDetail{invRole("r", map[string]string{"p": `not a policy`})}}},
+ "a group's inline policy that does not decode": {{GroupDetailList: []iamtypes.GroupDetail{invGroup("g", map[string]string{"p": `not a policy`})}}},
+ "a managed policy that does not decode": {{Policies: []iamtypes.ManagedPolicyDetail{invManaged("arn:aws:iam::123456789012:policy/p", `not a policy`)}}},
+ "a user in a group the listing did not return": {{UserDetailList: []iamtypes.UserDetail{invUser("u", []string{"ghosts"}, nil)}}},
+ "a group listed twice": {{GroupDetailList: []iamtypes.GroupDetail{invGroup("g", nil), invGroup("g", nil)}}},
+ "a principal with no ARN": {{RoleDetailList: []iamtypes.RoleDetail{{RoleName: aws.String("r")}}}},
+ } {
+ p, _ := inventoryFor(t, false, nil, pages...)
+ if grantees, err := p.BucketGrantees(ctx, bucket); err == nil {
+ t.Errorf("%s: %v with no error; want an error", name, grantees)
+ }
+ }
+ // A listing that never ends is given up on.
+ p, stub = inventoryFor(t, false, nil)
+ endless := &endlessInventory{}
+ p.inv = endless
+ if _, err := p.BucketGrantees(ctx, bucket); err == nil || endless.calls != inventoryMaxPages {
+ t.Fatalf("an endless listing: %v after %d pages", err, endless.calls)
+ }
+ _ = stub
+ // A name that is a pattern is not a name.
+ for _, name := range []string{"", "carlos-store-*", "a/b", "arn:aws:s3:::b", "a?b", "a${x}"} {
+ p, _ := inventoryFor(t, false, nil, good)
+ if _, err := p.BucketGrantees(ctx, name); err == nil {
+ t.Errorf("the name %q was answered for", name)
+ }
+ }
+}
+
+type endlessInventory struct{ calls int }
+
+func (e *endlessInventory) GetAccountAuthorizationDetails(context.Context, *iam.GetAccountAuthorizationDetailsInput, ...func(*iam.Options)) (*iam.GetAccountAuthorizationDetailsOutput, error) {
+ e.calls++
+ return &iam.GetAccountAuthorizationDetailsOutput{IsTruncated: true, Marker: aws.String(fmt.Sprint("m", e.calls))}, nil
+}
+
+// A declared platform principal is one principal, spelled out.
+func TestPlatformPrincipalsAreDeclaredOneByOne(t *testing.T) {
+ for _, arn := range []string{
+ "arn:aws:iam::123456789012:role/carlos-flagship-host",
+ "arn:aws:iam::123456789012:user/ops/someone",
+ "arn:aws:iam::123456789012:group/admins",
+ "arn:aws-us-gov:iam::123456789012:role/x",
+ } {
+ if err := validPlatformPrincipal(arn); err != nil {
+ t.Errorf("%s: %v", arn, err)
+ }
+ }
+ for _, arn := range []string{
+ "", "*", "carlos-flagship-host",
+ "arn:aws:iam::123456789012:role/*",
+ "arn:aws:iam::123456789012:role/carlos-*",
+ "arn:aws:iam::*:role/carlos-flagship-host",
+ "arn:aws:iam::123456789012:root",
+ "arn:aws:iam::123456789012:policy/p",
+ "arn:aws:iam::123456789012:role/",
+ "arn:aws:sts::123456789012:assumed-role/r/s",
+ "arn:aws:iam::12345:role/r",
+ "arn:aws:iam:eu-west-1:123456789012:role/r",
+ "arn:aws:s3:::carlos-store-bac-blog",
+ "arn:aws:iam::123456789012:role/${aws:username}",
+ } {
+ if err := validPlatformPrincipal(arn); err == nil {
+ t.Errorf("%q was accepted as a platform principal", arn)
+ }
+ }
+ if _, err := withInventory(&awsProvisioner{}, &stubInventory{}, []string{"arn:aws:iam::123456789012:role/*"}); err == nil {
+ t.Error("a pattern was accepted by the constructor")
+ }
+}
+
+// The plain AWS provisioner is not a GranteeProber: until the inventory is
+// constructed on purpose, the grant walk on AWS claims nothing, as before.
+func TestThePlainAWSProvisionerHasNoInventory(t *testing.T) {
+ var plain Provisioner = &awsProvisioner{}
+ if _, ok := plain.(GranteeProber); ok {
+ t.Fatal("the plain AWS provisioner answers for grantees")
+ }
+ if _, ok := plain.(BucketProber); !ok {
+ t.Fatal("the plain AWS provisioner lost its bucket probe")
+ }
+ var with Provisioner = &awsInventoryProvisioner{awsProvisioner: &awsProvisioner{}}
+ if _, ok := with.(GranteeProber); !ok {
+ t.Fatal("the inventory provisioner does not answer for grantees")
+ }
+ if _, ok := with.(BucketProber); !ok {
+ t.Fatal("the inventory provisioner lost the bucket probe")
+ }
+}
diff --git a/internal/objstore/provisioner.go b/internal/objstore/provisioner.go
index 59e1c3b29..36ed050c2 100644
--- a/internal/objstore/provisioner.go
+++ b/internal/objstore/provisioner.go
@@ -61,10 +61,13 @@ type BucketProber interface {
// An implementation that cannot answer for all of these returns an error,
// and an error claims nothing.
//
-// The AWS provisioner does not implement it yet: it needs to list the store
-// users and read their policies, which the console's role is not granted
-// today. Until it does, no bucket's ownership ledger claims completeness on
-// AWS, and store transfer holds.
+// The AWS provisioner the console constructs does not implement it. An
+// implementation for identity policies exists (principals.go: every user,
+// role and group, for a name no bucket holds) and is constructed by
+// nothing: it needs a listing the console's role is not granted, and a
+// decision on which principals are the platform's own. Until both, no
+// bucket's ownership ledger claims completeness on AWS, and store transfer
+// holds.
type GranteeProber interface {
BucketGrantees(ctx context.Context, name string) ([]string, error)
}