carlos / platform Public

The principal report: the third review's findings (every S3 resource that is not the bucket is a possible reach, incomplete ARNs as IAM reads them, version ids unique, condition values checked, a group's possible reach carried to its members, the label renamed)

Opus's first reading of 0e1bc298 (accept with fixes; two P2 required)
and Astra's third (reject; four P2, two P3). Answered here:

- The possible-reach rule is inverted: any S3 resource that is not a
  general-purpose bucket's (an access point, a multi-region access
  point, an Access Grants instance or grant, a Batch Operations job, an
  Object Lambda access point) is a possible reach of every bucket; the
  one exclusion is a resource type that holds no objects (Storage Lens).
- An incomplete ARN is read as IAM reads it, with the fields it leaves
  out as stars, by the direct matcher and the possible-reach rule alike.
- A managed policy's version ids must be present and unique.
- A null Id or Sid, a null condition operator, and a condition value
  that is not a scalar or a list of scalars are errors, in identity and
  trust policies, Deny statements included.
- A member of a group whose only reach is possible reaches possibly.
- The label "platform" is "named_by_deployment", with a legend for what
  each AssumableBy spelling means; the contract comment in
  provisioner.go no longer contradicts itself; the attestation plan
  carries a banner that its IAM-read completeness claim is superseded;
  the note says there is no harness to run the report yet.

Twenty more mutants in a tree copy over this round; all die (the full
list is in the note). Nothing was run against AWS; nothing is wired.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

claude 3 weeks ago 434cdf488e4d3b880b0df1800bbd99f12d8d22a1 parent 0e1bc29

5 files changed, +244 −79

docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md +3 −0
docs/superpowers/plans/2026-10-02-aws-principal-inventory.md +7 −7
internal/objstore/principals.go +148 −32
internal/objstore/principals_test.go +84 −38
internal/objstore/provisioner.go +2 −2

CI passed — run details

diff --git a/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md b/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md

index e64f0b429..2cb905898 100644

--- a/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md

+++ b/docs/superpowers/plans/2026-10-01-bucket-ownership-attestation.md

@@ -1,5 +1,8 @@

# Attesting the ownership history of existing and adopted store buckets

+> **Superseded in part, 2026-10-03.** "What can be proved" §2 and "The mechanism" step 2 below say that reading IAM now gives the complete set of principals that can reach a bucket, and that a ledger can claim completeness from the instant of that read. That is not so: a listing of the account is paged and eventually consistent, a role's policy is not everyone who can act as it, and an absent bucket is not one this grant creates (`2026-10-02-aws-principal-inventory.md`, which supersedes those two passages). The principal report on that branch supports a person's investigation and cannot vouch for an attestation. What a completeness claim needs instead is set out there, as proposals for the owner (decision 14).

Date: 2026-10-01. Status: proposal with the evidence it would rest on; not built. It adds a billing authority path, so it follows the review of everything already on the branch.

## The problem

diff --git a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md

index 43dcda596..2e0e7c8d1 100644

--- a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md

+++ b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md

@@ -1,6 +1,6 @@

# The AWS principal report, and what a completeness claim needs

-Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3) and on 2026-10-03 after its second (Astra: reject; six P2s about the report's own completeness and the two proposals; the Opus review has not run). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS.

+Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) and its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS.

## Why this exists

@@ -16,18 +16,18 @@ So the branch now does less, and says so.

## What is built

-`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (a grant on an access point, a multi-region access point or an Object Lambda access point, which names no bucket and can front any bucket's objects; marked so, never dropped), through which policies, and for each role, who its trust policy lets assume it. Principals the deployment names as its own are set apart in the report. That is a label for the reader and exempts nothing.

+`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, an Object Lambda access point; marked so, never dropped; a resource type that holds no objects, Storage Lens, is the one exclusion), through which policies, a group's possible reach carried to its members as possible, and for each role, who its trust policy lets assume it, with a legend for what each spelling means. Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. An incomplete ARN is read as IAM reads it, with the fields it leaves out as stars.

- It is **not** a `GranteeProber`, and a test says so. The grant walk on AWS claims nothing, as before. It is a report to support a person's investigation; it cannot verify that any rule is in force in the account, and identical reports can come from an account with and without one.

- Within the documents it is given it leans toward listing: a Deny, a condition or a boundary is not evaluated; `NotAction`, `NotResource` and policy variables are taken to reach.

-- What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a `Principal` in an identity policy, an unknown element, a condition or a sid of the wrong shape, an unknown principal kind in a trust policy); the same principal, managed policy or inline policy name returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing.

-- It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs may do under their roles; who inside the account may assume a role (`sts:AssumeRole` in their own policies); anything in another account; organization policies and permissions boundaries (which only narrow).

+- What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a null `Id` or `Sid`, a condition that is not a map of operators each mapping keys to scalars or lists of scalars, a `Principal` in an identity policy, an unknown element, an unknown principal kind in a trust policy); the same principal, managed policy, inline policy name or policy version id returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing.

+- It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs, or an Access Grants location role, may do under their roles, beyond reporting the principal that may start them as a possible reach; who inside the account may assume a role (`sts:AssumeRole` in their own policies) or pass it to a service (`iam:PassRole`); credentials vended by other services (Lake Formation, Identity Center grantees of Access Grants that are not IAM principals); anything in another account; organization policies and permissions boundaries (which only narrow).

-Use: a person preparing the attestation of the four existing stores, or checking a deployment, can see who reaches a store name today. It needs `iam:GetAccountAuthorizationDetails`, which the console's role does not have and this branch does not add; it can be run with an operator's own credentials.

+Use: a person investigating who reaches a store name today, in support of the decisions below. It is not evidence for an attestation: the attestation plan's claim that an IAM read proves completeness is superseded (a banner at its head says so). There is no harness to run it yet: an operator would need a small Go program in this module that loads their own profile, builds an IAM client, calls `NewPrincipalReport` and `BucketPrincipals` with a deadline, and holds `iam:GetAccountAuthorizationDetails`; from a sandboxed session that also needs the sandbox off. Nothing in the console calls it.

-Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format.

+Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. The direct matcher reads a resource pattern as text and the possible-reach rule reads it field by field; which of the two IAM itself does is on the list to verify against a real account.

-Mutation checks, in a tree copy: forty after the first review (thirty-nine die; one is equivalent, dropping the check that a default version has a document, because an absent document then fails to decode), and twelve after the second (repeated members not checked, and not inside a statement; the default-version id not compared; an access point grant, and an Object Lambda grant, nothing; a trust Deny not read in full; an unknown principal kind a label; a condition, and a sid, of any shape; a trust statement with no action; a possible reach read as direct; a star spanning fields no access path), all twelve die. Output: `inventory-mutants-round1.txt`, `inventory-mutants-round2.txt` in the run directory's logs.

+Mutation checks, in a tree copy: forty after the first review (thirty-nine die; one is equivalent, dropping the check that a default version has a document, because an absent document then fails to decode); twelve after the second (all die); twenty after the third (the earlier twelve again on the changed code, plus an incomplete ARN read literally, any S3 resource with a region or account read as nothing, a possible group reach propagated as direct, a version id given twice, a condition value of any shape, a null sid, Storage Lens a possible reach, a condition that is not a map, an Object Lambda grant nothing, a non-ARN pattern no access path; all die). Output: `inventory-mutants-round{1,2,3}.txt` in the run directory's logs.

## What a completeness claim needs instead (not built; needs the owner)

diff --git a/internal/objstore/principals.go b/internal/objstore/principals.go

index 118fc97cf..86616f7aa 100644

--- a/internal/objstore/principals.go

+++ b/internal/objstore/principals.go

@@ -78,9 +78,11 @@ import (

// and its access points' policies; what Batch Operations and replication

// jobs may do under their roles; anything in another account.

-// The platform's own. A deployment may name principals that are its own,

-// by exact ARN; the report sets them apart. That is a label for the

-// reader. It exempts nothing from anything.

+// Named by the deployment. A deployment may name principals as its own,

+// by exact ARN; the report sets them apart under that name. It is the

+// deployment's word, a label for the reader; it exempts nothing from

+// anything, and a role so named may be assumable by anyone its trust

+// policy lets in.

// IAMAuthorizationLister is the one IAM read the inventory makes.

// *iam.Client implements it.

@@ -103,21 +105,31 @@ type PrincipalGrant struct {

// of its groups that does. Sorted.

Via []string `json:"via"`

// AssumableBy, for a role, is every principal its trust policy's

- // Allow statements name ("AWS:<arn or account>", "Service:<name>",

- // "Federated:<provider>", or "*"), sorted: who can become this role

- // as far as the role itself says. It is not who can in fact.

+ // Allow statements name, sorted: who can become this role as far as

+ // the role itself says, not who can in fact. Read each spelling as:

+ // "AWS:arn:...:root" any principal in that account that may

+ // sts:AssumeRole, not the root user alone;

+ // "AWS:<arn>" that principal, if its own policy allows it;

+ // "Service:<name>" that service, on behalf of whoever may

+ // iam:PassRole this role to it;

+ // "Federated:<arn>" anyone that identity provider vouches for;

+ // "*" anyone the role's conditions, if any, let in.

AssumableBy []string `json:"assumable_by,omitempty"`

}

// PrincipalInventory is everything one listing showed about one name. It

-// is an observation, not a claim of completeness.

+// is an observation, not a claim of completeness, and nothing in it is

+// verified against anything but the listing.

type PrincipalInventory struct {

Bucket string `json:"bucket"`

// Reaching are the principals whose identity policies reach the name

// and that the deployment did not name as its own.

Reaching []PrincipalGrant `json:"reaching"`

- // Platform are those it did name.

- Platform []PrincipalGrant `json:"platform"`

+ // NamedByDeployment are the principals that reach the name and that

+ // the deployment's configuration names as its own. The name is the

+ // deployment's word, unverified: a role here may be assumable by

+ // anyone its AssumableBy says.

+ NamedByDeployment []PrincipalGrant `json:"named_by_deployment"`

// What was read, so an empty answer can be told from an empty account.

Users, Roles, Groups, ManagedPolicies int

}

@@ -289,11 +301,17 @@ func (p *PrincipalReport) listAccount(ctx context.Context) (*accountPolicies, er

defaults := 0

var current *string

var currentID string

+ versions := map[string]bool{}

for _, v := range m.PolicyVersionList {

+ id := aws.ToString(v.VersionId)

+ if id == "" || versions[id] {

+ return nil, fmt.Errorf("%w: managed policy %s version %q", errInventoryRepeated, arn, id)

+ }

+ versions[id] = true

if v.IsDefaultVersion {

defaults++

current = v.Document

- currentID = aws.ToString(v.VersionId)

+ currentID = id

}

if defaults != 1 || current == nil {

@@ -326,7 +344,7 @@ func (p *PrincipalReport) listAccount(ctx context.Context) (*accountPolicies, er

// reaching evaluates every principal against one bucket name.

func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*PrincipalInventory, error) {

- inv := &PrincipalInventory{Bucket: bucket, Reaching: []PrincipalGrant{}, Platform: []PrincipalGrant{}, Users: len(a.users), Roles: len(a.roles), Groups: len(a.groups), ManagedPolicies: len(a.managed)}

+ inv := &PrincipalInventory{Bucket: bucket, Reaching: []PrincipalGrant{}, NamedByDeployment: []PrincipalGrant{}, Users: len(a.users), Roles: len(a.roles), Groups: len(a.groups), ManagedPolicies: len(a.managed)}

via := func(pr inventoryPrincipal) ([]string, error) {

partition := "aws"

if parts := strings.SplitN(pr.arn, ":", 3); len(parts) == 3 && parts[0] == "arn" && parts[1] != "" {

@@ -338,7 +356,7 @@ func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*Pr

case reachDirect:

out = append(out, what)

case reachPossible:

- out = append(out, "possibly, through an access point or Object Lambda: "+what)

+ out = append(out, possiblyPrefix+what)

}

for name, doc := range pr.inline {

@@ -362,7 +380,7 @@ func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*Pr

sort.Strings(through)

g := PrincipalGrant{ARN: pr.arn, Kind: pr.kind, Via: through, AssumableBy: pr.assumableBy}

if platform[pr.arn] {

- inv.Platform = append(inv.Platform, g)

+ inv.NamedByDeployment = append(inv.NamedByDeployment, g)

} else {

inv.Reaching = append(inv.Reaching, g)

}

@@ -388,8 +406,22 @@ func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*Pr

if !ok {

return nil, fmt.Errorf("objstore: %s is in the group %q, and the listing did not return it", u.arn, name)

}

- if len(groupVia[name]) > 0 {

+ if len(groupVia[name]) == 0 {

+ continue

+ }

+ // The member reaches as the group does: if every one of the

+ // group's reaching policies is only a possible reach, so is

+ // the membership.

+ direct := false

+ for _, w := range groupVia[name] {

+ if !strings.HasPrefix(w, possiblyPrefix) {

+ direct = true

+ }

+ if direct {

through = append(through, "group:"+arn)

+ } else {

+ through = append(through, possiblyPrefix+"group:"+arn)

}

add(u, through)

@@ -401,7 +433,7 @@ func (a *accountPolicies) reaching(bucket string, platform map[string]bool) (*Pr

}

add(r, through)

}

- for _, list := range []*[]PrincipalGrant{&inv.Reaching, &inv.Platform} {

+ for _, list := range []*[]PrincipalGrant{&inv.Reaching, &inv.NamedByDeployment} {

sort.Slice(*list, func(i, j int) bool { return (*list)[i].ARN < (*list)[j].ARN })

}

return inv, nil

@@ -535,8 +567,8 @@ func policyText(raw string) (map[string]json.RawMessage, error) {

}

if v, ok := doc["Id"]; ok {

- var id string

- if err := json.Unmarshal(v, &id); err != nil {

+ var id *string

+ if err := json.Unmarshal(v, &id); err != nil || id == nil {

return nil, fmt.Errorf("policy document has the id %s", string(v))

}

@@ -557,15 +589,27 @@ func statementShape(i int, st map[string]json.RawMessage, allowed map[string]boo

}

switch key {

case "Sid":

- var sid string

- if err := json.Unmarshal(raw, &sid); err != nil {

+ var sid *string

+ if err := json.Unmarshal(raw, &sid); err != nil || sid == nil {

return fmt.Errorf("statement %d has the sid %s", i, string(raw))

}

case "Condition":

+ // Operators, each a map of keys to a scalar or a list of

+ // scalars. The meaning is not evaluated; the shape is.

var cond map[string]map[string]json.RawMessage

if err := json.Unmarshal(raw, &cond); err != nil || cond == nil {

return fmt.Errorf("statement %d has a condition that is not a map of operators", i)

}

+ for op, keys := range cond {

+ if keys == nil {

+ return fmt.Errorf("statement %d condition %s is null", i, op)

+ }

+ for k, v := range keys {

+ if !conditionValue(v) {

+ return fmt.Errorf("statement %d condition %s %s has a value that is not a scalar or a list of scalars", i, op, k)

+ }

}

return nil

@@ -597,6 +641,29 @@ func policyStatements(doc map[string]json.RawMessage) ([]map[string]json.RawMess

return list, nil

}

+// conditionValue reports whether a condition value is a string, a number,

+// a boolean, or a list of those.

+func conditionValue(raw json.RawMessage) bool {

+ var scalar any

+ if err := json.Unmarshal(raw, &scalar); err != nil {

+ return false

+ }

+ switch v := scalar.(type) {

+ case string, float64, bool:

+ return true

+ case []any:

+ for _, e := range v {

+ switch e.(type) {

+ case string, float64, bool:

+ default:

+ return false

+ }

+ return true

+ }

+ return false

func statementEffect(i int, st map[string]json.RawMessage) (allow bool, err error) {

var effect string

if raw, ok := st["Effect"]; !ok || json.Unmarshal(raw, &effect) != nil || (effect != "Allow" && effect != "Deny") {

@@ -767,6 +834,11 @@ func trustPrincipals(raw string) ([]string, error) {

return out, nil

}

+// possiblyPrefix marks, in a Via entry, a reach through an S3 resource

+// that is not the bucket itself and whose relation to the bucket the

+// report cannot see.

+const possiblyPrefix = "possibly, through an S3 resource that is not the bucket (an access point, an Access Grants instance, a job): "

// reachKind is how a document reaches a bucket: directly (the bucket's

// ARN or an object under it), or possibly, through a resource that can

// front a bucket's objects and whose relation to this bucket the report

@@ -821,28 +893,69 @@ func (d *policyDoc) reaches(partition, bucket string) reachKind {

return kind

}

-// resourceIsAnAccessPath reports whether a resource pattern can name an

-// access point (arn:<p>:s3:<region>:<account>:accesspoint/...), a

-// multi-region access point (arn:<p>:s3::<account>:accesspoint/...) or an

-// Object Lambda access point (arn:<p>:s3-object-lambda:...:accesspoint/...),

-// through which a bucket's objects can be reached. Which bucket is not in

-// the ARN, so any of them is a possible reach of every bucket. The ARN is

-// read field by field; a pattern with too few fields (a star standing for

-// several) is taken to be able to name one.

+// arnFields splits a resource pattern into the six fields of an ARN,

+// filling the fields an incomplete ARN leaves out with stars, as IAM does

+// (a pattern "arn:aws:s3" names every S3 resource). A pattern that does

+// not begin "arn:" is returned as nil.

+func arnFields(pattern string) []string {

+ if !strings.HasPrefix(pattern, "arn:") && pattern != "arn" {

+ return nil

+ }

+ parts := strings.SplitN(pattern, ":", 6)

+ for len(parts) < 6 {

+ parts = append(parts, "*")

+ }

+ return parts

+// resourceTypesThatHoldNoObjects are S3 resource types, by the word before

+// the first slash of the resource field, through which no object can be

+// read or written: a grant on one of them reaches no bucket's objects.

+// Everything else the S3 service names apart from a general-purpose

+// bucket (an access point, a multi-region access point, an Access Grants

+// instance and its grants, a Batch Operations job, a Storage Lens group's

+// data... ) can front a bucket's objects through a relation the ARN does

+// not show, and is a possible reach.

+var resourceTypesThatHoldNoObjects = map[string]bool{"storage-lens": true, "storage-lens-group": true}

+// resourceIsAnAccessPath reports whether a resource pattern can name an S3

+// resource that is not a general-purpose bucket or an object in one (an

+// access point, a multi-region access point, an Access Grants instance or

+// grant, a Batch Operations job, an Object Lambda access point), through

+// which a bucket's objects can be reached. Which bucket is not in the ARN,

+// so any of them is a possible reach of every bucket. The ARN is read

+// field by field, with the fields an incomplete ARN leaves out taken as

+// stars; a pattern that is not an ARN at all but can match anything ("*",

+// or a star before the first colon) is a possible reach too.

func resourceIsAnAccessPath(pattern, partition string, s3, lambda bool) bool {

p := conservativePattern(pattern)

- parts := strings.SplitN(p, ":", 6)

- if len(parts) < 6 {

+ parts := arnFields(p)

+ if parts == nil {

return strings.Contains(p, "*")

}

if !wildcardMatch(parts[0], "arn") || !wildcardMatch(parts[1], partition) {

return false

}

- accessPoint := wildcardLive(parts[5], "accesspoint/") != nil

- if !accessPoint {

+ svcS3 := s3 && wildcardMatch(parts[2], "s3")

+ svcLambda := lambda && wildcardMatch(parts[2], "s3-object-lambda")

+ if !svcS3 && !svcLambda {

+ return false

+ }

+ if svcLambda && !svcS3 {

+ // Every Object Lambda resource is an access point.

+ return true

+ }

+ // A general-purpose bucket's ARN has an empty region and an empty

+ // account; the direct matcher decides those. Any other S3 resource is

+ // a possible reach, unless its type holds no objects.

+ if parts[3] == "" && parts[4] == "" {

+ return false

+ }

+ resourceType, _, _ := strings.Cut(parts[5], "/")

+ if !strings.ContainsAny(resourceType, "*?") && resourceTypesThatHoldNoObjects[resourceType] {

return false

}

- return (s3 && wildcardMatch(parts[2], "s3")) || (lambda && wildcardMatch(parts[2], "s3-object-lambda"))

+ return true

}

// actionCanBeObjectLambda reports whether an action pattern can name an

@@ -873,6 +986,9 @@ func actionCanBeS3(pattern string) bool {

// ARN, or can match the ARN of some object under it.

func resourceReaches(pattern, bucketARN string) bool {

p := conservativePattern(pattern)

+ if parts := arnFields(p); parts != nil {

+ p = strings.Join(parts, ":")

+ }

if wildcardMatch(p, bucketARN) {

return true

}

diff --git a/internal/objstore/principals_test.go b/internal/objstore/principals_test.go

index 34c3cdf62..0733d8ee7 100644

--- a/internal/objstore/principals_test.go

+++ b/internal/objstore/principals_test.go

@@ -155,17 +155,18 @@ func TestPolicyPatternsThatReachABucket(t *testing.T) {

doc string

want bool

}{

- "NotResource on an Allow": {`{"Statement":[{"Effect":"Allow","Action":"s3:GetObject","NotResource":"arn:aws:s3:::carlos-store-bac-blog/*"}]}`, true},

- "NotAction on an Allow": {`{"Statement":[{"Effect":"Allow","NotAction":"s3:*","Resource":"*"}]}`, true},

- "NotAction with another bucket": {`{"Statement":[{"Effect":"Allow","NotAction":"iam:*","Resource":"arn:aws:s3:::other"}]}`, false},

- "a Deny alone": {`{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, false},

- "an Allow that a Deny takes back": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"},{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, true},

- "an Allow under a condition": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"Bool":{"aws:MultiFactorAuthPresent":"true"}}}]}`, true},

- "one statement, not a list": {`{"Statement":{"Effect":"Allow","Action":"s3:*","Resource":"*"}}`, true},

- "the older version": {`{"Version":"2008-10-17","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},

- "a statement id and a policy id": {`{"Version":"2012-10-17","Id":"p","Statement":[{"Sid":"one","Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},

- "an escaped star, taken as any": {`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:GetObject","Resource":"arn:aws:s3:::carlos-store-bac-${*}/*"}]}`, true},

- "the second of two statements": {`{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"},{"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::carlos-store-bac-blog"}]}`, true},

+ "NotResource on an Allow": {`{"Statement":[{"Effect":"Allow","Action":"s3:GetObject","NotResource":"arn:aws:s3:::carlos-store-bac-blog/*"}]}`, true},

+ "NotAction on an Allow": {`{"Statement":[{"Effect":"Allow","NotAction":"s3:*","Resource":"*"}]}`, true},

+ "NotAction with another bucket": {`{"Statement":[{"Effect":"Allow","NotAction":"iam:*","Resource":"arn:aws:s3:::other"}]}`, false},

+ "a Deny alone": {`{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, false},

+ "an Allow that a Deny takes back": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"},{"Effect":"Deny","Action":"s3:*","Resource":"*"}]}`, true},

+ "an Allow under a condition": {`{"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"Bool":{"aws:MultiFactorAuthPresent":"true"}}}]}`, true},

+ "one statement, not a list": {`{"Statement":{"Effect":"Allow","Action":"s3:*","Resource":"*"}}`, true},

+ "the older version": {`{"Version":"2008-10-17","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},

+ "a condition with a list of scalars": {`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"*","Condition":{"StringEquals":{"aws:username":["a","b"]},"Bool":{"aws:SecureTransport":true},"NumericLessThan":{"s3:max-keys":10}}}]}`, true},

+ "a statement id and a policy id": {`{"Version":"2012-10-17","Id":"p","Statement":[{"Sid":"one","Effect":"Allow","Action":"s3:*","Resource":"*"}]}`, true},

+ "an escaped star, taken as any": {`{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:GetObject","Resource":"arn:aws:s3:::carlos-store-bac-${*}/*"}]}`, true},

+ "the second of two statements": {`{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"},{"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::carlos-store-bac-blog"}]}`, true},

} {

doc, err := decodePolicy(tc.doc)

if err != nil {

@@ -204,6 +205,11 @@ func TestPolicyPatternsThatReachABucket(t *testing.T) {

"a member given twice inside a statement": `{"Statement":[{"Effect":"Allow","Action":"s3:*","Action":"ec2:*","Resource":"*"}]}`,

"a condition that is a number": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":7}]}`,

"a condition that is a list": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":["x"]}]}`,

+ "an id that is null": `{"Id":null,"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,

+ "a sid that is null": `{"Statement":[{"Sid":null,"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,

+ "a condition operator that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":null}}]}`,

+ "a condition value that is an object": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":{"unexpected":"object"}}}}]}`,

+ "a condition value that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":null}}}]}`,

"a sid that is a number": `{"Statement":[{"Sid":1,"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,

"an id that is a number": `{"Id":1,"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`,

"a Deny with an unknown element": `{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*","Unexpected":true}]}`,

@@ -277,7 +283,7 @@ func TestInventoryListsEveryPrincipalThatReachesTheName(t *testing.T) {

if got := arns(inv.Reaching); !reflect.DeepEqual(got, want) {

t.Fatalf("reaching\n %v\nwant\n %v", got, want)

}

- if len(inv.Platform) != 0 || inv.Users != 6 || inv.Groups != 2 || inv.Roles != 3 || inv.ManagedPolicies != 3 {

+ if len(inv.NamedByDeployment) != 0 || inv.Users != 6 || inv.Groups != 2 || inv.Roles != 3 || inv.ManagedPolicies != 3 {

t.Fatalf("inventory %+v", inv)

}

if !reflect.DeepEqual(stub.calls, []string{"", "m1"}) {

@@ -313,7 +319,7 @@ func TestInventoryListsEveryPrincipalThatReachesTheName(t *testing.T) {

if err != nil {

t.Fatal(err)

}

- if got := arns(inv.Platform); !reflect.DeepEqual(got, []string{platform[1], platform[0]}) {

+ if got := arns(inv.NamedByDeployment); !reflect.DeepEqual(got, []string{platform[1], platform[0]}) {

t.Fatalf("platform %v", got)

}

if got := arns(inv.Reaching); !reflect.DeepEqual(got, []string{want[0], want[3], want[4], want[5]}) {

@@ -324,7 +330,7 @@ func TestInventoryListsEveryPrincipalThatReachesTheName(t *testing.T) {

t.Fatalf("a new name under the prefix: %v %v; want the prefix readers, who are nobody's named own", inv, err)

}

p, _ = inventoryFor(t, platform, page1, page2)

- if inv, err := p.BucketPrincipals(context.Background(), "elsewhere-entirely"); err != nil || len(inv.Reaching) != 0 || len(inv.Platform) != 1 {

+ if inv, err := p.BucketPrincipals(context.Background(), "elsewhere-entirely"); err != nil || len(inv.Reaching) != 0 || len(inv.NamedByDeployment) != 1 {

t.Fatalf("a name only the administrator reaches: %+v %v", inv, err)

}

@@ -369,20 +375,21 @@ func TestInventoryReportsWhoARoleTrusts(t *testing.T) {

}

// A trust policy it cannot read is an error, not a role with nobody.

for name, doc := range map[string]string{

- "no principal": `{"Statement":[{"Effect":"Allow","Action":"sts:AssumeRole"}]}`,

- "a principal that is a word": `{"Statement":[{"Effect":"Allow","Principal":"everyone","Action":"sts:AssumeRole"}]}`,

- "an empty principal map": `{"Statement":[{"Effect":"Allow","Principal":{},"Action":"sts:AssumeRole"}]}`,

- "a null principal value": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":null},"Action":"sts:AssumeRole"}]}`,

- "no trust policy": ``,

- "an unknown effect": `{"Statement":[{"Effect":"Maybe","Principal":"*"}]}`,

- "a Deny with an unknown element": `{"Statement":{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Unexpected":true}}`,

- "a Deny with a null principal": `{"Statement":{"Effect":"Deny","Principal":null,"Action":"sts:AssumeRole"}}`,

- "an Allow with a null action": `{"Statement":[{"Effect":"Allow","Principal":"*","Action":null}]}`,

- "an Allow with no action": `{"Statement":[{"Effect":"Allow","Principal":"*"}]}`,

- "an unknown principal kind": `{"Statement":[{"Effect":"Allow","Principal":{"Anyone":"*"},"Action":"sts:AssumeRole"}]}`,

- "a NotPrincipal that is null": `{"Statement":[{"Effect":"Allow","NotPrincipal":null,"Action":"sts:AssumeRole"}]}`,

- "both Principal and NotPrincipal": `{"Statement":[{"Effect":"Allow","Principal":"*","NotPrincipal":{"AWS":"x"},"Action":"sts:AssumeRole"}]}`,

- "a member given twice": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::999999999999:root"},"Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}`,

+ "no principal": `{"Statement":[{"Effect":"Allow","Action":"sts:AssumeRole"}]}`,

+ "a principal that is a word": `{"Statement":[{"Effect":"Allow","Principal":"everyone","Action":"sts:AssumeRole"}]}`,

+ "an empty principal map": `{"Statement":[{"Effect":"Allow","Principal":{},"Action":"sts:AssumeRole"}]}`,

+ "a null principal value": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":null},"Action":"sts:AssumeRole"}]}`,

+ "no trust policy": ``,

+ "an unknown effect": `{"Statement":[{"Effect":"Maybe","Principal":"*"}]}`,

+ "a Deny with an unknown element": `{"Statement":{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Unexpected":true}}`,

+ "a Deny with a null principal": `{"Statement":{"Effect":"Deny","Principal":null,"Action":"sts:AssumeRole"}}`,

+ "an Allow with a null action": `{"Statement":[{"Effect":"Allow","Principal":"*","Action":null}]}`,

+ "an Allow with no action": `{"Statement":[{"Effect":"Allow","Principal":"*"}]}`,

+ "an unknown principal kind": `{"Statement":[{"Effect":"Allow","Principal":{"Anyone":"*"},"Action":"sts:AssumeRole"}]}`,

+ "a NotPrincipal that is null": `{"Statement":[{"Effect":"Allow","NotPrincipal":null,"Action":"sts:AssumeRole"}]}`,

+ "both Principal and NotPrincipal": `{"Statement":[{"Effect":"Allow","Principal":"*","NotPrincipal":{"AWS":"x"},"Action":"sts:AssumeRole"}]}`,

+ "a member given twice": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::999999999999:root"},"Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}`,

+ "a Deny with a malformed condition": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{"StringEquals":{"aws:username":{"x":1}}}}]}`,

} {

p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{role("r", doc)}})

if inv, err := p.BucketPrincipals(context.Background(), "carlos-store-bac-blog"); err == nil {

@@ -415,6 +422,13 @@ func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) {

"an S3 action on an access point in another partition": {allow(`"s3:GetObject"`, `"arn:aws-cn:s3:cn-north-1:123456789012:accesspoint/readers/object/*"`), reachNone},

"an EC2 action on an access point ARN": {allow(`"ec2:*"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers"`), reachNone},

"S3 on Outposts": {allow(`"s3-outposts:GetObject"`, `"arn:aws:s3-outposts:eu-west-1:123456789012:outpost/op-1/accesspoint/ap/object/*"`), reachNone},

+ "an Access Grants instance": {allow(`"s3:GetDataAccess"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default"`), reachPossible},

+ "an Access Grants grant": {allow(`"s3:CreateAccessGrant"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default/grant/*"`), reachPossible},

+ "a Batch Operations job": {allow(`"s3:CreateJob"`, `"arn:aws:s3:eu-west-1:123456789012:job/*"`), reachPossible},

+ "a Storage Lens configuration": {allow(`"s3:GetStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/x"`), reachNone},

+ "an incomplete ARN for all of S3": {allow(`"s3:GetObject"`, `"arn:aws:s3"`), reachDirect},

+ "an incomplete ARN for all of Object Lambda": {allow(`"s3-object-lambda:GetObject"`, `"arn:aws:s3-object-lambda"`), reachPossible},

+ "an incomplete ARN for another service": {allow(`"s3:GetObject"`, `"arn:aws:lambda"`), reachNone},

} {

doc, err := decodePolicy(tc.doc)

if err != nil {

@@ -425,13 +439,31 @@ func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) {

t.Errorf("%s: reaches %v, want %v", name, got, tc.want)

}

- // In the report: set apart in the Via, never dropped.

- p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{UserDetailList: []iamtypes.UserDetail{

- invUser("through-an-access-point", nil, map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}),

- }})

+ // In the report: set apart in the Via, never dropped; and a member of

+ // a group whose only reach is possible reaches possibly too.

+ p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{

+ UserDetailList: []iamtypes.UserDetail{

+ invUser("through-an-access-point", nil, map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}),

+ invUser("in-the-ap-group", []string{"ap-readers"}, nil),

+ invUser("in-the-direct-group", []string{"direct-readers"}, nil),

+ },

+ GroupDetailList: []iamtypes.GroupDetail{

+ invGroup("ap-readers", map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}),

+ invGroup("direct-readers", map[string]string{"d": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)}),

+ },

+ })

inv, err := p.BucketPrincipals(context.Background(), bucket)

- if err != nil || len(inv.Reaching) != 1 || len(inv.Reaching[0].Via) != 1 || !strings.HasPrefix(inv.Reaching[0].Via[0], "possibly, through an access point or Object Lambda: inline:ap") {

- t.Fatalf("the report: %+v %v", inv, err)

+ if err != nil {

+ t.Fatal(err)

+ }

+ via := map[string][]string{}

+ for _, g := range inv.Reaching {

+ via[g.ARN] = g.Via

+ }

+ if !reflect.DeepEqual(via[invAccount+"user/through-an-access-point"], []string{possiblyPrefix + "inline:ap"}) ||

+ !reflect.DeepEqual(via[invAccount+"user/in-the-ap-group"], []string{possiblyPrefix + "group:" + invAccount + "group/ap-readers"}) ||

+ !reflect.DeepEqual(via[invAccount+"user/in-the-direct-group"], []string{"group:" + invAccount + "group/direct-readers"}) {

+ t.Fatalf("the report: %v", via)

}

@@ -514,22 +546,36 @@ func TestInventoryClaimsNothingItCannotRead(t *testing.T) {

"a group twice": {{GroupDetailList: []iamtypes.GroupDetail{invGroup("g", nil), invGroup("g", nil)}}},

"an inline policy name twice": {{UserDetailList: []iamtypes.UserDetail{{Arn: aws.String(invAccount + "user/u"), UserName: aws.String("u"), UserPolicyList: []iamtypes.PolicyDetail{

{PolicyName: aws.String("p"), PolicyDocument: encoded(reaches)}, {PolicyName: aws.String("p"), PolicyDocument: encoded(harmless)}}}}}},

+ "a managed policy whose default version has no id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{{IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

+ "one version id twice": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{

+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v1"), Document: encoded(reaches)}}}}}},

+ "a version with no id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{

+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {Document: encoded(reaches)}}}}}},

} {

p, _ := inventoryFor(t, nil, pages...)

if inv, err := p.BucketPrincipals(ctx, bucket); !errors.Is(err, errInventoryRepeated) {

t.Errorf("%s: %+v %v; want errInventoryRepeated", name, inv, err)

}

+ // The two-default refusal is that refusal, not a later one.

+ for _, name := range []string{"a managed policy with two default versions", "two default versions, the first with no document"} {

+ _ = name

+ }

+ twoDefaults := &iam.GetAccountAuthorizationDetailsOutput{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{

+ {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}

+ p, _ = inventoryFor(t, nil, twoDefaults)

+ if _, err := p.BucketPrincipals(ctx, bucket); err == nil || !strings.Contains(err.Error(), "default versions") {

+ t.Fatalf("two default versions: %v; want that refusal by name", err)

+ }

for name, pages := range map[string][]*iam.GetAccountAuthorizationDetailsOutput{

"a managed policy attached and not listed": {{RoleDetailList: []iamtypes.RoleDetail{invRole("r", nil, "arn:aws:iam::123456789012:policy/missing")}}},

"a managed policy with no default version": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), Document: encoded(harmless)}}}}}},

"a managed policy whose default id names another version": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{

{VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), Document: encoded(reaches)}}}}}},

- "a managed policy with no default id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

- "a managed policy whose default version has no id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v1"), PolicyVersionList: []iamtypes.PolicyVersion{{IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

- "a managed policy with two default versions": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{

+ "a managed policy with no default id": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

+ "a managed policy with two default versions": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{

{VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

- "two default versions, the first with no document": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{

+ "two default versions, the first with no document": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{

{VersionId: aws.String("v1"), IsDefaultVersion: true}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}}},

"a default version with no document": {{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), PolicyVersionList: []iamtypes.PolicyVersion{{VersionId: aws.String("v1"), IsDefaultVersion: true}}}}}},

"an inline policy that does not decode": {{UserDetailList: []iamtypes.UserDetail{invUser("u", nil, map[string]string{"p": `{"Statement":[{"Effect":"Allow","Action":"s3:*"}]}`})}}},

diff --git a/internal/objstore/provisioner.go b/internal/objstore/provisioner.go

index 5ad371362..0394f770c 100644

--- a/internal/objstore/provisioner.go

+++ b/internal/objstore/provisioner.go

@@ -47,8 +47,8 @@ type BucketProber interface {

// An absent bucket proves nothing about credentials: a key scoped to a

// bucket name outlives the bucket (a teardown that deleted the bucket and

// then failed to delete the user), and reaches a bucket of that name the

-// moment it is created again. Only an inventory of principals can show

-// that nobody is waiting on a name.

+// moment it is created again. No listing of principals can show that

+// nobody is waiting on a name either; see below.

// What an implementation must cover, or it must not exist: every kind of

// principal (users, roles and groups, not only the store users this package