amadan
The principal report: the fourth review's findings (a pattern read as text and as padded fields, nothing S3 names excluded, empty conditions refused, the trust legend corrected)
Opus P2-1: reading an incomplete ARN only as padded fields dropped grants
the text reading reported ("arn:*:carlos-store-bac-blog" on
s3:PutBucketPolicy read as a service called by the bucket's name). Both
matchers now read a pattern as text and, when it is an incomplete ARN, as
padded fields, and report a reach under either; which reading IAM gives
is on the verify list, and the over-report the text reading costs (an
incomplete ARN of another service whose star runs into an object key) is
pinned by a test.
Astra 1: the Storage Lens exclusion dropped a grant through which objects
can be written (a configuration exports into a bucket of its owner's
choosing). Nothing S3 names is excluded now: every S3 resource that is
not a general-purpose bucket or an object in one is a possible reach.
Astra 2: an empty condition, an operator with no key and an empty list of
values were read as a successful report. Each is an error, in identity
policies and in trust policies, Deny statements included.
Opus P2-2 and Astra 3: the AssumableBy legend said a named principal needs
its own policy (not in the same account) and that a service means its
PassRole holders (not whoever runs code where the role already runs).
Reworded, with the header's and the note's "does not read" lists
extended to control of compute running as a listed role and to IAM
writes.
Opus P3s: the header and reachKind comment name the inverted rule; the
note's matcher sentence is current; the dead loop is gone; the contract
says "an AWS account's principals"; proposal A names the management
account and resource control policies; the counts carry JSON tags. A
group with a direct and a possible reach is tested to carry direct. The
CUR collector plan and OUTSTANDING item 2 no longer name an inventory in
the provisioner as the missing mechanism. Ten mutants in a tree copy, all
die (inventory-mutants-round4.txt).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 files changed,
+147
−83
docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md+1 −1docs/superpowers/plans/2026-10-02-aws-principal-inventory.md+9 −7internal/objstore/principals.go+97 −61internal/objstore/principals_test.go+38 −12internal/objstore/provisioner.go+2 −2
CI passed — run details
diff --git a/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md b/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md| index 86084da53..0167676d4 100644 |
| --- a/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md |
| +++ b/docs/superpowers/plans/2026-10-01-store-transfer-cur-collector.md |
| @@ -45,7 +45,7 @@ Verification after apply: within 24 hours `billing/cur/carloku-transfer/metadata |
| - The raw result is `logs/aws-ce-s3-2026-07..09.json` in the run directory. It was one Cost Explorer request, which AWS bills at one cent. |
| - **Credits and discounts hold the month.** AWS writes `Credit`, `Refund`, `Fee`, `EdpDiscount`, `PrivateRateDiscount` and `BundledDiscount` lines on the same transfer usage types as the usage they adjust. Any such line with a non-zero cost holds store transfer for the month for every account (`transfer_adjustment`), because AWS then charged something other than what the usage lines say, and nothing lets an operator release that hold. None appeared from July to September 2026 (above). Check again when the export is approved, and at the payer account, where an organization's credits are applied: if one ever touches S3 transfer the meter will not complete as built, and the classifier needs a ruling on how a discounted byte is charged on. |
| - **A store's first clock hour never resolves.** A ledger answers for hours that begin at or after the instant it began vouching. A charged line in the hour a store was minted in has no provable owner and holds the month, like any other unowned line. With no free allowance in this account, a store that serves a download in its first hour does that. Nothing releases such a hold today. The choices are an operator release door for a named bucket-hour, or a ruling that the first partial hour of a new store is the platform's cost. That is a product decision; until it is made the code holds. |
| -- **Existing stores on AWS never complete yet.** A ledger claims completeness only when the provisioner can list every principal whose policy names the bucket and finds none. The AWS provisioner cannot list principals today (the console role has no IAM list permission), so on AWS no ledger claims completeness, new stores included, and store transfer holds. The principal inventory is the first step of the ownership attestation plan (`2026-10-01-bucket-ownership-attestation.md`). |
| +- **Existing stores on AWS never complete yet.** A ledger claims completeness only when the provisioner can list every principal whose policy names the bucket and finds none. On AWS no listing can show that (`2026-10-02-aws-principal-inventory.md`: a listing is not an instant, a policy is not everyone who can act as its principal, an absent bucket is not one this grant creates), so no ledger on AWS claims completeness, new stores included, and store transfer holds until the account enforces a rule and the bucket's creation is provably the grant's own (proposals A and B there). The principal report built on that branch supports a person's investigation; it is not the mechanism. |
| - The statements catch-up revisits every month in a 36-month window that is open, was never written, or is missing an account's statement. It costs one read per account per metered month in a console's first pass and nothing after a statement has been seen closed. |
| - The per-month rebuild door for a statement closed before this existed is still missing (requirements audit step 1). |
diff --git a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md| index 2e0e7c8d1..756bfb7bd 100644 |
| --- a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md |
| +++ b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md |
| @@ -1,6 +1,6 @@ |
| # The AWS principal report, and what a completeness claim needs |
| -Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) and its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS. |
| +Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s) and its fourth (Opus: accept with fixes, two P2s required; Astra: reject, three P2s). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS. |
| ## Why this exists |
| @@ -16,24 +16,24 @@ So the branch now does less, and says so. |
| ## What is built |
| -`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, an Object Lambda access point; marked so, never dropped; a resource type that holds no objects, Storage Lens, is the one exclusion), through which policies, a group's possible reach carried to its members as possible, and for each role, who its trust policy lets assume it, with a legend for what each spelling means. Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. An incomplete ARN is read as IAM reads it, with the fields it leaves out as stars. |
| +`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, a Storage Lens configuration, an Object Lambda access point, a type it does not know; marked so, never dropped; nothing S3 names is excluded, since a Storage Lens configuration exports into a bucket of its owner's choosing), through which policies, a group's possible reach carried to its members as possible, and for each role, the principal values its trust policy's Allow statements name, with a legend for what each spelling means (not who can in fact assume it, which the report does not determine). Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. A resource pattern is read two ways, as text (a star free to run across colons) and as the six fields of an ARN with the ones an incomplete pattern leaves out as stars, and a reach under either is reported; which reading IAM gives an incomplete pattern is unverified. The legend lives in the Go type's comment; the JSON carries none, and whatever prints the report for a person must print it. |
| - It is **not** a `GranteeProber`, and a test says so. The grant walk on AWS claims nothing, as before. It is a report to support a person's investigation; it cannot verify that any rule is in force in the account, and identical reports can come from an account with and without one. |
| - Within the documents it is given it leans toward listing: a Deny, a condition or a boundary is not evaluated; `NotAction`, `NotResource` and policy variables are taken to reach. |
| -- What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a null `Id` or `Sid`, a condition that is not a map of operators each mapping keys to scalars or lists of scalars, a `Principal` in an identity policy, an unknown element, an unknown principal kind in a trust policy); the same principal, managed policy, inline policy name or policy version id returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing. |
| -- It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs, or an Access Grants location role, may do under their roles, beyond reporting the principal that may start them as a possible reach; who inside the account may assume a role (`sts:AssumeRole` in their own policies) or pass it to a service (`iam:PassRole`); credentials vended by other services (Lake Formation, Identity Center grantees of Access Grants that are not IAM principals); anything in another account; organization policies and permissions boundaries (which only narrow). |
| +- What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a null `Id` or `Sid`, a condition that is not a map of at least one operator each mapping at least one key to a scalar or a non-empty list of scalars, a `Principal` in an identity policy, an unknown element, an unknown principal kind in a trust policy); the same principal, managed policy, inline policy name or policy version id returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing. |
| +- It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs, or an Access Grants location role, may do under their roles, beyond reporting the principal that may start them as a possible reach; who inside the account may assume a role (`sts:AssumeRole` in their own policies), pass it to a service (`iam:PassRole`) or run code where it already runs (a host under SSM or Instance Connect, a function's code, a container: everyone who controls compute running as a listed role reaches what it reaches); who may change IAM (`iam:PutUserPolicy`, `iam:AttachRolePolicy`, `iam:CreatePolicyVersion` and the like) and so grant themselves the bucket in one call; credentials vended by other services (Lake Formation, Identity Center grantees of Access Grants that are not IAM principals); anything in another account; organization policies and permissions boundaries (which only narrow). |
| Use: a person investigating who reaches a store name today, in support of the decisions below. It is not evidence for an attestation: the attestation plan's claim that an IAM read proves completeness is superseded (a banner at its head says so). There is no harness to run it yet: an operator would need a small Go program in this module that loads their own profile, builds an IAM client, calls `NewPrincipalReport` and `BucketPrincipals` with a deadline, and holds `iam:GetAccountAuthorizationDetails`; from a sandboxed session that also needs the sandbox off. Nothing in the console calls it. |
| -Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. The direct matcher reads a resource pattern as text and the possible-reach rule reads it field by field; which of the two IAM itself does is on the list to verify against a real account. |
| +Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. Both matchers read a resource pattern as text and, when it is an incomplete ARN, as padded fields too, and report a reach under either; which reading IAM gives an incomplete pattern, and whether its wildcards run across colons, is on the list to verify against a real account. The text reading over-reports: an incomplete ARN of another service with a star can match an object key, and is reported as a direct reach. |
| -Mutation checks, in a tree copy: forty after the first review (thirty-nine die; one is equivalent, dropping the check that a default version has a document, because an absent document then fails to decode); twelve after the second (all die); twenty after the third (the earlier twelve again on the changed code, plus an incomplete ARN read literally, any S3 resource with a region or account read as nothing, a possible group reach propagated as direct, a version id given twice, a condition value of any shape, a null sid, Storage Lens a possible reach, a condition that is not a map, an Object Lambda grant nothing, a non-ARN pattern no access path; all die). Output: `inventory-mutants-round{1,2,3}.txt` in the run directory's logs. |
| +Mutation checks, in a tree copy: forty after the first review (thirty-nine die; one is equivalent, dropping the check that a default version has a document, because an absent document then fails to decode); twelve after the second (all die); twenty after the third (the earlier twelve again on the changed code, plus an incomplete ARN read literally, any S3 resource with a region or account read as nothing, a possible group reach propagated as direct, a version id given twice, a condition value of any shape, a null sid, Storage Lens a possible reach, a condition that is not a map, an Object Lambda grant nothing, a non-ARN pattern no access path; all die); ten after the fourth (the text reading dropped from the direct matcher, the padded reading dropped, the text reading dropped from the possible-reach rule and applied to one service only, Storage Lens excluded again, an empty condition, an operator with no key and an empty value list accepted, a group with a direct and a possible reach propagated as possible; all die). Output: `inventory-mutants-round{1,2,3,4}.txt` in the run directory's logs. |
| ## What a completeness claim needs instead (not built; needs the owner) |
| Looking cannot establish it. A rule the account enforces can. Two parts, each a design question, each stated here with what its review found missing in it: |
| -**A. Nobody else can reach a store bucket.** Proposed direction: an organization policy (or, failing that, a permissions boundary required on every principal the platform creates plus an account rule) that denies S3 actions on the store bucket prefix to every principal except each bucket's own store user and a short list named by ARN. What that needs, beyond the sentence: a binding between each store identity and its one bucket, enforced (a store user under the path with a policy for another store's bucket must be impossible, not merely absent); coverage of the paths an identity policy does not show (resource policies and grants to other accounts, service-linked roles, access points); the enforcement configuration itself protected from the principals it constrains; and a way to verify the control is in force that is separate from this report, which cannot see it. Who is on the named list, and who can assume each role on it, is decision 14. |
| +**A. Nobody else can reach a store bucket.** Proposed direction: an organization policy (or, failing that, a permissions boundary required on every principal the platform creates plus an account rule) that denies S3 actions on the store bucket prefix to every principal except each bucket's own store user and a short list named by ARN. What that needs, beyond the sentence: a binding between each store identity and its one bucket, enforced (a store user under the path with a policy for another store's bucket must be impossible, not merely absent); coverage of the paths an identity policy does not show (resource policies and grants to other accounts, service-linked roles, access points); the enforcement configuration itself protected from the principals it constrains; and a way to verify the control is in force that is separate from this report, which cannot see it. An organization policy binds neither the organization's management account nor service-linked roles, so if the flagship account is the management account A cannot stand on one; a resource control policy on the buckets is the nearer tool, and which account the flagship is sits on the list below. Who is on the named list, and who can assume each role on it, is decision 14. |
| **B. The bucket is the one this grant made.** Proposed direction: the grant mints a bucket name with a random part recorded in its grant block before any provider call, and refuses a bucket that already exists under that name. What that needs, beyond the sentence: randomness only makes an accidental collision and a guess unlikely; an administrator who can read the grant block, or the store user's policy written before the bucket, can create the name first, so creation provenance needs an enforceable restriction on who may create buckets under the prefix, not unpredictability; an explicit treatment of the provider answering an already-owned bucket's create as success (the us-east-1 case the provisioner already notes) and of a retry after the platform's own create; and crash recovery that tells the platform's own earlier create from someone else's. |
| @@ -50,3 +50,5 @@ Proposed: only the deployment's host role and roles that need account administra |
| 1. The owner's answers to item 14 and to A and B. |
| 2. A real listing, read by a person: encodings, service-linked roles, Identity Center roles, AWS managed policies with constructs this parser refuses. The first real run may error. |
| 3. The resource side for existing buckets (bucket policy, ACL, access points) and the attestation door. |
| +4. IAM's own matching, which this parser guesses at: whether an incomplete resource ARN is padded, read as text or refused when the policy is written; whether `*` runs across colons; whether the service field of a resource ARN is case-folded; and whether real documents carry shapes this parser refuses (an empty condition, an operator with no key, an empty list of values), which would make the first real run an error rather than a report. |
| +5. Whether the flagship account is its organization's management account, which decides whether proposal A can stand on an organization policy. |
diff --git a/internal/objstore/principals.go b/internal/objstore/principals.go| index 86616f7aa..6f9fa8126 100644 |
| --- a/internal/objstore/principals.go |
| +++ b/internal/objstore/principals.go |
| @@ -29,10 +29,12 @@ import ( |
| // complete, and reading it twice does not make it so. |
| // - A principal's policy is not everyone who can act as it. A role can |
| // be assumed by whoever its trust policy names, another account's |
| -// principals included, and by whoever may pass it to a service. The |
| -// trust policy's principals are reported for each role that reaches |
| -// the name; who in turn may assume it from inside this account |
| -// (sts:AssumeRole in their own policies) is not traced. |
| +// principals included; by whoever may pass it to a service; and by |
| +// whoever can run code where it already runs (a host under SSM, a |
| +// function's code, a container). Whoever can write IAM can grant |
| +// themselves the bucket in one call. The trust policy's principal |
| +// values are reported for each role that reaches the name; none of |
| +// the rest is traced. |
| // - An absent bucket now is not a bucket this grant creates. Another |
| // actor in the account can create the name, with a policy of its own, |
| // between any probe and the walk's own create, and AWS answers the |
| @@ -60,23 +62,37 @@ import ( |
| // expand to anything. |
| // - A group whose policy reaches the bucket is listed, with or without |
| // members, and so is each of its members. |
| -// - A grant on an access point, a multi-region access point or an |
| -// Object Lambda access point names no bucket in its ARN and can front |
| -// any bucket's objects: its principal is listed as possibly reaching, |
| -// marked so, never dropped. |
| +// - A grant on any S3 resource that is not a general-purpose bucket or |
| +// an object in one (an access point, a multi-region access point, an |
| +// Object Lambda access point, an Access Grants instance, a job, a |
| +// Storage Lens configuration, a type it does not know) names no |
| +// bucket in its ARN and can reach a bucket's objects through a |
| +// relation the ARN does not show: its principal is listed as possibly |
| +// reaching, marked so, never dropped. |
| +// - A resource pattern is read two ways, as text and as the six fields |
| +// of an ARN with the ones an incomplete pattern leaves out as stars; |
| +// a reach under either is reported. Which reading IAM gives an |
| +// incomplete pattern is unverified. |
| // |
| // What it cannot read is an error, never a shorter list: a document it |
| // cannot decode; a JSON member given twice anywhere in a document; a |
| // statement or an element that is not the grammar it knows, in an |
| -// identity policy or a trust policy, Deny statements included; the same |
| -// principal, policy or inline policy name returned twice; a managed |
| +// identity policy or a trust policy, Deny statements included (a |
| +// condition with no operator, an operator with no key and a key with an |
| +// empty list of values among them); the same principal, policy or inline |
| +// policy name returned twice; a managed |
| // policy whose own default-version id and the version marked default |
| // disagree; an attached managed policy the listing did not return; a |
| // listing truncated without a marker, or that does not end. |
| // |
| // What it does not read at all: the root user; a bucket's own policy, ACL |
| // and its access points' policies; what Batch Operations and replication |
| -// jobs may do under their roles; anything in another account. |
| +// jobs, or an Access Grants location role, may do under their roles; who |
| +// may assume a role from inside the account, pass it to a service, or |
| +// control compute that runs as it; who may change IAM, and so grant |
| +// themselves the bucket; credentials other services vend (Lake Formation, |
| +// Access Grants to grantees that are not IAM principals); anything in |
| +// another account. |
| // |
| // Named by the deployment. A deployment may name principals as its own, |
| // by exact ARN; the report sets them apart under that name. It is the |
| @@ -104,14 +120,19 @@ type PrincipalGrant struct { |
| // "managed:<policy arn>", and for a user "group:<group arn>" for each |
| // of its groups that does. Sorted. |
| Via []string `json:"via"` |
| - // AssumableBy, for a role, is every principal its trust policy's |
| - // Allow statements name, sorted: who can become this role as far as |
| - // the role itself says, not who can in fact. Read each spelling as: |
| + // AssumableBy, for a role, is every principal value its trust |
| + // policy's Allow statements name, sorted, whatever their actions and |
| + // conditions: not who can in fact assume the role, which the report |
| + // does not determine. Read each spelling as: |
| // "AWS:arn:...:root" any principal in that account that may |
| // sts:AssumeRole, not the root user alone; |
| - // "AWS:<arn>" that principal, if its own policy allows it; |
| - // "Service:<name>" that service, on behalf of whoever may |
| - // iam:PassRole this role to it; |
| + // "AWS:<arn>" that principal; in this account the trust |
| + // policy alone is enough, from another account |
| + // its own policy must allow it too; |
| + // "Service:<name>" that service, and so whoever may iam:PassRole |
| + // this role to it and whoever can run code |
| + // where it already runs (a host under SSM, a |
| + // function's code, a container); |
| // "Federated:<arn>" anyone that identity provider vouches for; |
| // "*" anyone the role's conditions, if any, let in. |
| AssumableBy []string `json:"assumable_by,omitempty"` |
| @@ -131,7 +152,10 @@ type PrincipalInventory struct { |
| // anyone its AssumableBy says. |
| NamedByDeployment []PrincipalGrant `json:"named_by_deployment"` |
| // What was read, so an empty answer can be told from an empty account. |
| - Users, Roles, Groups, ManagedPolicies int |
| + Users int `json:"users"` |
| + Roles int `json:"roles"` |
| + Groups int `json:"groups"` |
| + ManagedPolicies int `json:"managed_policies"` |
| } |
| // PrincipalReport reads the account and reports on bucket names. |
| @@ -594,15 +618,16 @@ func statementShape(i int, st map[string]json.RawMessage, allowed map[string]boo |
| return fmt.Errorf("statement %d has the sid %s", i, string(raw)) |
| } |
| case "Condition": |
| - // Operators, each a map of keys to a scalar or a list of |
| - // scalars. The meaning is not evaluated; the shape is. |
| + // At least one operator, each a map of at least one key to a |
| + // scalar or a non-empty list of scalars. The meaning is not |
| + // evaluated; the shape is. |
| var cond map[string]map[string]json.RawMessage |
| - if err := json.Unmarshal(raw, &cond); err != nil || cond == nil { |
| - return fmt.Errorf("statement %d has a condition that is not a map of operators", i) |
| + if err := json.Unmarshal(raw, &cond); err != nil || len(cond) == 0 { |
| + return fmt.Errorf("statement %d has a condition that is not a map of at least one operator", i) |
| } |
| for op, keys := range cond { |
| - if keys == nil { |
| - return fmt.Errorf("statement %d condition %s is null", i, op) |
| + if len(keys) == 0 { |
| + return fmt.Errorf("statement %d condition %s is null or has no key", i, op) |
| } |
| for k, v := range keys { |
| if !conditionValue(v) { |
| @@ -642,7 +667,7 @@ func policyStatements(doc map[string]json.RawMessage) ([]map[string]json.RawMess |
| } |
| // conditionValue reports whether a condition value is a string, a number, |
| -// a boolean, or a list of those. |
| +// a boolean, or a list of at least one of those. |
| func conditionValue(raw json.RawMessage) bool { |
| var scalar any |
| if err := json.Unmarshal(raw, &scalar); err != nil { |
| @@ -652,6 +677,9 @@ func conditionValue(raw json.RawMessage) bool { |
| case string, float64, bool: |
| return true |
| case []any: |
| + if len(v) == 0 { |
| + return false |
| + } |
| for _, e := range v { |
| switch e.(type) { |
| case string, float64, bool: |
| @@ -837,13 +865,15 @@ func trustPrincipals(raw string) ([]string, error) { |
| // possiblyPrefix marks, in a Via entry, a reach through an S3 resource |
| // that is not the bucket itself and whose relation to the bucket the |
| // report cannot see. |
| -const possiblyPrefix = "possibly, through an S3 resource that is not the bucket (an access point, an Access Grants instance, a job): " |
| +const possiblyPrefix = "possibly, through an S3 resource that is not the bucket (an access point, an Access Grants instance, a job, a Storage Lens configuration): " |
| // reachKind is how a document reaches a bucket: directly (the bucket's |
| -// ARN or an object under it), or possibly, through a resource that can |
| -// front a bucket's objects and whose relation to this bucket the report |
| -// cannot see (an access point, a multi-region access point, an Object |
| -// Lambda access point). A possible reach is reported, never dropped. |
| +// ARN or an object under it), or possibly, through any S3 resource that |
| +// is not a general-purpose bucket or an object in one, whose relation to |
| +// this bucket the report cannot see (an access point, a multi-region |
| +// access point, an Object Lambda access point, an Access Grants instance, |
| +// a job, a Storage Lens configuration, a type it does not know). A |
| +// possible reach is reported, never dropped. |
| type reachKind int |
| const ( |
| @@ -894,9 +924,11 @@ func (d *policyDoc) reaches(partition, bucket string) reachKind { |
| } |
| // arnFields splits a resource pattern into the six fields of an ARN, |
| -// filling the fields an incomplete ARN leaves out with stars, as IAM does |
| -// (a pattern "arn:aws:s3" names every S3 resource). A pattern that does |
| -// not begin "arn:" is returned as nil. |
| +// filling the fields an incomplete ARN leaves out with stars: one reading |
| +// of an incomplete pattern (under it "arn:aws:s3" names every S3 |
| +// resource). The other reading is the text itself, a star free to run |
| +// across colons; the matchers try both, because which reading IAM gives |
| +// is unverified. A pattern that does not begin "arn:" is returned as nil. |
| func arnFields(pattern string) []string { |
| if !strings.HasPrefix(pattern, "arn:") && pattern != "arn" { |
| return nil |
| @@ -908,31 +940,30 @@ func arnFields(pattern string) []string { |
| return parts |
| } |
| -// resourceTypesThatHoldNoObjects are S3 resource types, by the word before |
| -// the first slash of the resource field, through which no object can be |
| -// read or written: a grant on one of them reaches no bucket's objects. |
| -// Everything else the S3 service names apart from a general-purpose |
| -// bucket (an access point, a multi-region access point, an Access Grants |
| -// instance and its grants, a Batch Operations job, a Storage Lens group's |
| -// data... ) can front a bucket's objects through a relation the ARN does |
| -// not show, and is a possible reach. |
| -var resourceTypesThatHoldNoObjects = map[string]bool{"storage-lens": true, "storage-lens-group": true} |
| - |
| // resourceIsAnAccessPath reports whether a resource pattern can name an S3 |
| // resource that is not a general-purpose bucket or an object in one (an |
| // access point, a multi-region access point, an Access Grants instance or |
| -// grant, a Batch Operations job, an Object Lambda access point), through |
| -// which a bucket's objects can be reached. Which bucket is not in the ARN, |
| -// so any of them is a possible reach of every bucket. The ARN is read |
| -// field by field, with the fields an incomplete ARN leaves out taken as |
| -// stars; a pattern that is not an ARN at all but can match anything ("*", |
| -// or a star before the first colon) is a possible reach too. |
| +// grant, a Batch Operations job, a Storage Lens configuration, an Object |
| +// Lambda access point, a type this report does not know), through which |
| +// a bucket's objects can be read or written by a relation the ARN does |
| +// not show (a Storage Lens configuration, for one, exports into a bucket |
| +// of its owner's choosing). Which bucket is not in the ARN, so any of |
| +// them is a possible reach of every bucket. The ARN is read field by |
| +// field, with the fields an incomplete ARN leaves out taken as stars; an |
| +// incomplete ARN is read as text as well, a star free to run across |
| +// colons (for a complete one the two readings differ only through an |
| +// object key holding colons, which the direct matcher already reports); |
| +// a pattern that is not an ARN at all but can match anything ("*", or a |
| +// star before the first colon) is a possible reach too. |
| func resourceIsAnAccessPath(pattern, partition string, s3, lambda bool) bool { |
| p := conservativePattern(pattern) |
| parts := arnFields(p) |
| if parts == nil { |
| return strings.Contains(p, "*") |
| } |
| + if strings.Join(parts, ":") != p && ((s3 && wildcardLive(p, "arn:"+partition+":s3:") != nil) || (lambda && wildcardLive(p, "arn:"+partition+":s3-object-lambda:") != nil)) { |
| + return true |
| + } |
| if !wildcardMatch(parts[0], "arn") || !wildcardMatch(parts[1], partition) { |
| return false |
| } |
| @@ -947,15 +978,8 @@ func resourceIsAnAccessPath(pattern, partition string, s3, lambda bool) bool { |
| } |
| // A general-purpose bucket's ARN has an empty region and an empty |
| // account; the direct matcher decides those. Any other S3 resource is |
| - // a possible reach, unless its type holds no objects. |
| - if parts[3] == "" && parts[4] == "" { |
| - return false |
| - } |
| - resourceType, _, _ := strings.Cut(parts[5], "/") |
| - if !strings.ContainsAny(resourceType, "*?") && resourceTypesThatHoldNoObjects[resourceType] { |
| - return false |
| - } |
| - return true |
| + // a possible reach. |
| + return parts[3] != "" || parts[4] != "" |
| } |
| // actionCanBeObjectLambda reports whether an action pattern can name an |
| @@ -983,12 +1007,24 @@ func actionCanBeS3(pattern string) bool { |
| } |
| // resourceReaches reports whether a resource pattern matches the bucket's |
| -// ARN, or can match the ARN of some object under it. |
| +// ARN, or can match the ARN of some object under it, under either reading |
| +// of the pattern: as text, or as the fields of an ARN with the ones an |
| +// incomplete pattern leaves out as stars. |
| func resourceReaches(pattern, bucketARN string) bool { |
| p := conservativePattern(pattern) |
| - if parts := arnFields(p); parts != nil { |
| - p = strings.Join(parts, ":") |
| + readings := []string{p} |
| + if parts := arnFields(p); parts != nil && strings.Join(parts, ":") != p { |
| + readings = append(readings, strings.Join(parts, ":")) |
| + } |
| + for _, p := range readings { |
| + if resourceReadingReaches(p, bucketARN) { |
| + return true |
| + } |
| } |
| + return false |
| +} |
| + |
| +func resourceReadingReaches(p, bucketARN string) bool { |
| if wildcardMatch(p, bucketARN) { |
| return true |
| } |
diff --git a/internal/objstore/principals_test.go b/internal/objstore/principals_test.go| index 0733d8ee7..788c7d4f3 100644 |
| --- a/internal/objstore/principals_test.go |
| +++ b/internal/objstore/principals_test.go |
| @@ -210,6 +210,10 @@ func TestPolicyPatternsThatReachABucket(t *testing.T) { |
| "a condition operator that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":null}}]}`, |
| "a condition value that is an object": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":{"unexpected":"object"}}}}]}`, |
| "a condition value that is null": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":null}}}]}`, |
| + "an empty condition": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{}}]}`, |
| + "a condition operator with no key": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{}}}]}`, |
| + "a condition value that is an empty list": `{"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*","Condition":{"StringEquals":{"aws:username":[]}}}]}`, |
| + "a Deny with an empty condition": `{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*","Condition":{}}]}`, |
| "a sid that is a number": `{"Statement":[{"Sid":1,"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`, |
| "an id that is a number": `{"Id":1,"Statement":[{"Effect":"Allow","Action":"ec2:*","Resource":"*"}]}`, |
| "a Deny with an unknown element": `{"Statement":[{"Effect":"Deny","Action":"s3:*","Resource":"*","Unexpected":true}]}`, |
| @@ -390,6 +394,8 @@ func TestInventoryReportsWhoARoleTrusts(t *testing.T) { |
| "both Principal and NotPrincipal": `{"Statement":[{"Effect":"Allow","Principal":"*","NotPrincipal":{"AWS":"x"},"Action":"sts:AssumeRole"}]}`, |
| "a member given twice": `{"Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::999999999999:root"},"Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}`, |
| "a Deny with a malformed condition": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{"StringEquals":{"aws:username":{"x":1}}}}]}`, |
| + "a Deny with an empty condition": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{}}]}`, |
| + "a Deny whose operator has no key": `{"Statement":[{"Effect":"Deny","Principal":"*","Action":"sts:AssumeRole","Condition":{"StringEquals":{}}}]}`, |
| } { |
| p, _ := inventoryFor(t, nil, &iam.GetAccountAuthorizationDetailsOutput{RoleDetailList: []iamtypes.RoleDetail{role("r", doc)}}) |
| if inv, err := p.BucketPrincipals(context.Background(), "carlos-store-bac-blog"); err == nil { |
| @@ -398,10 +404,14 @@ func TestInventoryReportsWhoARoleTrusts(t *testing.T) { |
| } |
| } |
| -// A grant on an access point, a multi-region access point or an Object |
| -// Lambda access point names no bucket and can front any bucket's objects: |
| -// its principal is reported as possibly reaching, marked so. A grant on |
| -// another service's resource is not. |
| +// A grant on any S3 resource that is not a general-purpose bucket or an |
| +// object in one (an access point, a multi-region access point, an Object |
| +// Lambda access point, an Access Grants instance, a job, a Storage Lens |
| +// configuration) names no bucket and can reach a bucket's objects by a |
| +// relation the ARN does not show: its principal is reported as possibly |
| +// reaching, marked so. A grant on another service's resource is not. A |
| +// pattern is read as text and as padded fields, and a reach under either |
| +// is reported. |
| func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) { |
| const bucket = "carlos-store-bac-blog" |
| for name, tc := range map[string]struct { |
| @@ -425,10 +435,26 @@ func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) { |
| "an Access Grants instance": {allow(`"s3:GetDataAccess"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default"`), reachPossible}, |
| "an Access Grants grant": {allow(`"s3:CreateAccessGrant"`, `"arn:aws:s3:eu-west-1:123456789012:access-grants/default/grant/*"`), reachPossible}, |
| "a Batch Operations job": {allow(`"s3:CreateJob"`, `"arn:aws:s3:eu-west-1:123456789012:job/*"`), reachPossible}, |
| - "a Storage Lens configuration": {allow(`"s3:GetStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/x"`), reachNone}, |
| - "an incomplete ARN for all of S3": {allow(`"s3:GetObject"`, `"arn:aws:s3"`), reachDirect}, |
| - "an incomplete ARN for all of Object Lambda": {allow(`"s3-object-lambda:GetObject"`, `"arn:aws:s3-object-lambda"`), reachPossible}, |
| - "an incomplete ARN for another service": {allow(`"s3:GetObject"`, `"arn:aws:lambda"`), reachNone}, |
| + // (A Storage Lens configuration exports into a bucket of its |
| + // owner's choosing: whoever may write one may have objects written.) |
| + "reading a Storage Lens configuration": {allow(`"s3:GetStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/x"`), reachPossible}, |
| + "writing a Storage Lens configuration": {allow(`"s3:PutStorageLensConfiguration"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens/export"`), reachPossible}, |
| + "a Storage Lens group": {allow(`"s3:*"`, `"arn:aws:s3:eu-west-1:123456789012:storage-lens-group/g"`), reachPossible}, |
| + "an S3 resource type it does not know": {allow(`"s3:*"`, `"arn:aws:s3:eu-west-1:123456789012:something-new/x"`), reachPossible}, |
| + "an incomplete ARN for all of S3": {allow(`"s3:GetObject"`, `"arn:aws:s3"`), reachDirect}, |
| + "an incomplete ARN for all of Object Lambda": {allow(`"s3-object-lambda:GetObject"`, `"arn:aws:s3-object-lambda"`), reachPossible}, |
| + "an incomplete ARN for another service": {allow(`"s3:GetObject"`, `"arn:aws:lambda"`), reachNone}, |
| + // Read as padded fields these name a service called by the |
| + // bucket's name; read as text the star runs across the fields to |
| + // the bucket itself. Either reading reports. |
| + "a star that runs across the fields to the bucket": {allow(`"s3:PutBucketPolicy"`, `"arn:*:carlos-store-bac-blog"`), reachDirect}, |
| + "a star that runs across the fields to a prefix": {allow(`"s3:PutBucketPolicy"`, `"arn:*:carlos-store-*"`), reachDirect}, |
| + "a star that runs into the bucket's name": {allow(`"s3:PutBucketPolicy"`, `"arn:aws:*carlos-store-bac-blog"`), reachDirect}, |
| + "a star that runs across the fields to an access point": {allow(`"s3-object-lambda:GetObject"`, `"arn:*:accesspoint/thumbs"`), reachPossible}, |
| + // (And, read as text, an incomplete ARN of another service whose |
| + // star can run into an object key holding ":function:thumbs": a |
| + // direct reach, the over-report the text reading costs.) |
| + "an incomplete ARN of another service with a star": {allow(`"s3:GetObject"`, `"arn:*:function:thumbs"`), reachDirect}, |
| } { |
| doc, err := decodePolicy(tc.doc) |
| if err != nil { |
| @@ -446,10 +472,12 @@ func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) { |
| invUser("through-an-access-point", nil, map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}), |
| invUser("in-the-ap-group", []string{"ap-readers"}, nil), |
| invUser("in-the-direct-group", []string{"direct-readers"}, nil), |
| + invUser("in-the-mixed-group", []string{"mixed-readers"}, nil), |
| }, |
| GroupDetailList: []iamtypes.GroupDetail{ |
| invGroup("ap-readers", map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`)}), |
| invGroup("direct-readers", map[string]string{"d": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)}), |
| + invGroup("mixed-readers", map[string]string{"ap": allow(`"s3:GetObject"`, `"arn:aws:s3:eu-west-1:123456789012:accesspoint/readers/object/*"`), "d": allow(`"s3:GetObject"`, `"arn:aws:s3:::carlos-store-bac-blog/*"`)}), |
| }, |
| }) |
| inv, err := p.BucketPrincipals(context.Background(), bucket) |
| @@ -462,7 +490,8 @@ func TestAnAccessPointGrantIsAPossibleReach(t *testing.T) { |
| } |
| if !reflect.DeepEqual(via[invAccount+"user/through-an-access-point"], []string{possiblyPrefix + "inline:ap"}) || |
| !reflect.DeepEqual(via[invAccount+"user/in-the-ap-group"], []string{possiblyPrefix + "group:" + invAccount + "group/ap-readers"}) || |
| - !reflect.DeepEqual(via[invAccount+"user/in-the-direct-group"], []string{"group:" + invAccount + "group/direct-readers"}) { |
| + !reflect.DeepEqual(via[invAccount+"user/in-the-direct-group"], []string{"group:" + invAccount + "group/direct-readers"}) || |
| + !reflect.DeepEqual(via[invAccount+"user/in-the-mixed-group"], []string{"group:" + invAccount + "group/mixed-readers"}) { |
| t.Fatalf("the report: %v", via) |
| } |
| } |
| @@ -558,9 +587,6 @@ func TestInventoryClaimsNothingItCannotRead(t *testing.T) { |
| } |
| } |
| // The two-default refusal is that refusal, not a later one. |
| - for _, name := range []string{"a managed policy with two default versions", "two default versions, the first with no document"} { |
| - _ = name |
| - } |
| twoDefaults := &iam.GetAccountAuthorizationDetailsOutput{Policies: []iamtypes.ManagedPolicyDetail{{Arn: aws.String(managed), DefaultVersionId: aws.String("v2"), PolicyVersionList: []iamtypes.PolicyVersion{ |
| {VersionId: aws.String("v1"), IsDefaultVersion: true, Document: encoded(harmless)}, {VersionId: aws.String("v2"), IsDefaultVersion: true, Document: encoded(harmless)}}}}} |
| p, _ = inventoryFor(t, nil, twoDefaults) |
diff --git a/internal/objstore/provisioner.go b/internal/objstore/provisioner.go| index 0394f770c..4e26a00ba 100644 |
| --- a/internal/objstore/provisioner.go |
| +++ b/internal/objstore/provisioner.go |
| @@ -47,8 +47,8 @@ type BucketProber interface { |
| // An absent bucket proves nothing about credentials: a key scoped to a |
| // bucket name outlives the bucket (a teardown that deleted the bucket and |
| // then failed to delete the user), and reaches a bucket of that name the |
| -// moment it is created again. No listing of principals can show that |
| -// nobody is waiting on a name either; see below. |
| +// moment it is created again. No listing of an AWS account's |
| +// principals can show that nobody is waiting on a name either; see below. |
| // |
| // What an implementation must cover, or it must not exist: every kind of |
| // principal (users, roles and groups, not only the store users this package |