| index 756bfb7bd..c050927d8 100644 |
| --- a/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md |
| +++ b/docs/superpowers/plans/2026-10-02-aws-principal-inventory.md |
| @@ -1,6 +1,6 @@ |
| # The AWS principal report, and what a completeness claim needs |
| |
| -Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s) and its fourth (Opus: accept with fixes, two P2s required; Astra: reject, three P2s). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS. |
| +Date: 2026-10-02. Revised the same day after its first review (Astra: reject; three P1s, two P2s, one P3), and on 2026-10-03 after its second (Astra: reject; six P2s) its third (Opus: accept with fixes, two P2s required; Astra: reject, four P2s, two P3s) its fourth (Opus: accept with fixes, two P2s required; Astra: reject, three P2s) and its fifth (both accept with fixes: one P2 each on the possible-reach rule's text reading, and the host role's reach in decision 14). Status: a report for a person, built on `feat/carloku-principal-inventory-20261002`. **It is not wired, and it is no longer offered as the proof a bucket's ledger needs.** Nothing was run against AWS. |
| |
| ## Why this exists |
| |
| @@ -9,31 +9,31 @@ Object-store transfer is billed to a tenant only if the bucket's ownership ledge |
| The first build of this branch tried to make the claim by reading the account: list every principal, evaluate every policy, and call an empty answer "nobody else". The review showed that an empty answer from one listing is not that claim, for three reasons that no amount of care in the reading removes: |
| |
| 1. **A listing is not an instant.** IAM is eventually consistent: a policy attached before the listing began can be missing from it. The listing is paged: a principal on page one can be given access before the last page is read. Nothing in the answer says it is complete. |
| -2. **A principal's policy is not everyone who can act as it.** A role can be assumed by whoever its trust policy names, another account included, and by whoever may pass it to a service. Setting a role aside as "the platform's" sets aside everyone who can become it. |
| +2. **A principal's policy is not everyone who can act as it.** A role can be assumed by whoever its trust policy names, another account included; by whoever may pass it to a service or run code where it already runs; and whoever can change IAM can grant themselves the bucket. Setting a role aside as "the platform's" sets aside everyone who can become it. |
| 3. **An absent bucket is not a bucket this grant creates.** Another actor in the account can create the name, with a policy of its own, between any probe and the walk's own create; AWS answers the walk's create as a success. |
| |
| So the branch now does less, and says so. |
| |
| ## What is built |
| |
| -`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, a Storage Lens configuration, an Object Lambda access point, a type it does not know; marked so, never dropped; nothing S3 names is excluded, since a Storage Lens configuration exports into a bucket of its owner's choosing), through which policies, a group's possible reach carried to its members as possible, and for each role, the principal values its trust policy's Allow statements name, with a legend for what each spelling means (not who can in fact assume it, which the report does not determine). Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. A resource pattern is read two ways, as text (a star free to run across colons) and as the six fields of an ARN with the ones an incomplete pattern leaves out as stars, and a reach under either is reported; which reading IAM gives an incomplete pattern is unverified. The legend lives in the Go type's comment; the JSON carries none, and whatever prints the report for a person must print it. |
| +`internal/objstore/principals.go`: `PrincipalReport.BucketPrincipals(name)`. One listing of the account (`iam:GetAccountAuthorizationDetails`, every page), every identity policy decoded and evaluated against one bucket name, and a report: which users, roles and groups reach the name, directly (the bucket's ARN or an object under it) or possibly (any S3 resource that is not a general-purpose bucket's and whose relation to the bucket the ARN does not show: an access point, a multi-region access point, an Access Grants instance or grant, a Batch Operations job, a Storage Lens configuration, an Object Lambda access point, a type it does not know; marked so, never dropped; nothing S3 names is excluded, since a Storage Lens configuration exports into a bucket of its owner's choosing), through which policies, a group's possible reach carried to its members as possible, and for each role, the principal values its trust policy's Allow statements name, with a legend for what each spelling means (not who can in fact assume it, which the report does not determine). Principals the deployment names as its own are set apart as `named_by_deployment`: the deployment's word, unverified, exempting nothing. A resource pattern is read two ways, as text (a star or a question mark free to run across colons) and as the six fields of an ARN with the ones an incomplete pattern leaves out as stars, and a reach under either is reported; the possible-reach rule is one text test on both forms (can the pattern spell `arn:<partition>:s3:` or `arn:<partition>:s3::` and then a character other than a colon, or anything under `arn:<partition>:s3-object-lambda:`); which reading IAM gives an incomplete pattern is unverified. The legend lives in the Go type's comment; the JSON carries none, and whatever prints the report for a person must print it. |
| |
| - It is **not** a `GranteeProber`, and a test says so. The grant walk on AWS claims nothing, as before. It is a report to support a person's investigation; it cannot verify that any rule is in force in the account, and identical reports can come from an account with and without one. |
| - Within the documents it is given it leans toward listing: a Deny, a condition or a boundary is not evaluated; `NotAction`, `NotResource` and policy variables are taken to reach. |
| - What it cannot read is an error, never a shorter list: an undecodable document; a JSON member given twice anywhere in a document; a statement, element or version outside the grammar it knows, in an identity policy or a trust policy, Deny statements included (a null or empty `Statement`, an empty or null value, a null `Id` or `Sid`, a condition that is not a map of at least one operator each mapping at least one key to a scalar or a non-empty list of scalars, a `Principal` in an identity policy, an unknown element, an unknown principal kind in a trust policy); the same principal, managed policy, inline policy name or policy version id returned twice; a managed policy whose own default-version id and the version marked default disagree; an attached policy the listing did not return; a truncated or endless listing. |
| - It does not read: the root user; a bucket's own policy, ACL or its access points' own policies; what Batch Operations and replication jobs, or an Access Grants location role, may do under their roles, beyond reporting the principal that may start them as a possible reach; who inside the account may assume a role (`sts:AssumeRole` in their own policies), pass it to a service (`iam:PassRole`) or run code where it already runs (a host under SSM or Instance Connect, a function's code, a container: everyone who controls compute running as a listed role reaches what it reaches); who may change IAM (`iam:PutUserPolicy`, `iam:AttachRolePolicy`, `iam:CreatePolicyVersion` and the like) and so grant themselves the bucket in one call; credentials vended by other services (Lake Formation, Identity Center grantees of Access Grants that are not IAM principals); anything in another account; organization policies and permissions boundaries (which only narrow). |
| |
| -Use: a person investigating who reaches a store name today, in support of the decisions below. It is not evidence for an attestation: the attestation plan's claim that an IAM read proves completeness is superseded (a banner at its head says so). There is no harness to run it yet: an operator would need a small Go program in this module that loads their own profile, builds an IAM client, calls `NewPrincipalReport` and `BucketPrincipals` with a deadline, and holds `iam:GetAccountAuthorizationDetails`; from a sandboxed session that also needs the sandbox off. Nothing in the console calls it. |
| +Use: a person investigating who reaches a store name today, in support of the decisions below. It is not evidence for an attestation: the attestation plan's claim that an IAM read proves completeness is superseded (a banner at its head says so). There is no harness to run it yet: an operator would need a small Go program in this module that loads their own profile, builds an IAM client, calls `NewPrincipalReport` and `BucketPrincipals` with a deadline, and holds `iam:GetAccountAuthorizationDetails`; from a sandboxed session that also needs network and credential access the sandbox withholds. Nothing in the console calls it. |
| |
| -Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. Both matchers read a resource pattern as text and, when it is an incomplete ARN, as padded fields too, and report a reach under either; which reading IAM gives an incomplete pattern, and whether its wildcards run across colons, is on the list to verify against a real account. The text reading over-reports: an incomplete ARN of another service with a star can match an object key, and is reported as a direct reach. |
| +Tests: `go test ./internal/objstore -run 'TestPolicyPatterns|TestInventory|TestPlatformPrincipals|TestNothingOnAWS|TestAnAccessPoint'`, against a stub of the listing. No test reads a real account, and the stub is built from SDK structs, not from IAM's wire format. Both matchers read a resource pattern as text and, when it is an incomplete ARN, as padded fields too, and report a reach under either; which reading IAM gives an incomplete pattern, and whether its wildcards run across colons, is on the list to verify against a real account. The text reading over-reports, and in one common case relabels: any star before a literal an object key can hold runs into one of this bucket's keys, so a grant on `arn:aws:s3:*:123456789012:accesspoint/*` is listed as direct rather than possible, and an incomplete ARN of another service with a star (`arn:*:function:thumbs`) is listed as direct; a question mark cannot run into a key that way and is listed as possible. |
| |
| -Mutation checks, in a tree copy: forty after the first review (thirty-nine die; one is equivalent, dropping the check that a default version has a document, because an absent document then fails to decode); twelve after the second (all die); twenty after the third (the earlier twelve again on the changed code, plus an incomplete ARN read literally, any S3 resource with a region or account read as nothing, a possible group reach propagated as direct, a version id given twice, a condition value of any shape, a null sid, Storage Lens a possible reach, a condition that is not a map, an Object Lambda grant nothing, a non-ARN pattern no access path; all die); ten after the fourth (the text reading dropped from the direct matcher, the padded reading dropped, the text reading dropped from the possible-reach rule and applied to one service only, Storage Lens excluded again, an empty condition, an operator with no key and an empty value list accepted, a group with a direct and a possible reach propagated as possible; all die). Output: `inventory-mutants-round{1,2,3,4}.txt` in the run directory's logs. |
| +Mutation checks, in a tree copy, each a one-line edit of `principals.go` run against the inventory tests. The logs of the first four rounds (`inventory-mutants-round{1,2,3,4}.txt` in the run directory's logs) record for each mutation its description and whether the tests failed, not which test; their counts are as logged (forty after the first review, one equivalent; twelve after the second; the third's log holds the K, L and M entries with replacements for the ones that first failed to compile; ten after the fourth), and nothing more is claimed of them. The fifth round's log (`inventory-mutants-round5.txt`) records for each mutation the exact edit and the failing tests' names and first assertion, which is the evidence a kill needs; it reruns the fourth round's ten on the changed code and adds the possible-reach rule's text test (the S3 test dropped; the account form dropped; the non-colon requirement dropped; the padded reading dropped; the Object Lambda test dropped). Any that survive are named there. |
| |
| ## What a completeness claim needs instead (not built; needs the owner) |
| |
| Looking cannot establish it. A rule the account enforces can. Two parts, each a design question, each stated here with what its review found missing in it: |
| |
| -**A. Nobody else can reach a store bucket.** Proposed direction: an organization policy (or, failing that, a permissions boundary required on every principal the platform creates plus an account rule) that denies S3 actions on the store bucket prefix to every principal except each bucket's own store user and a short list named by ARN. What that needs, beyond the sentence: a binding between each store identity and its one bucket, enforced (a store user under the path with a policy for another store's bucket must be impossible, not merely absent); coverage of the paths an identity policy does not show (resource policies and grants to other accounts, service-linked roles, access points); the enforcement configuration itself protected from the principals it constrains; and a way to verify the control is in force that is separate from this report, which cannot see it. An organization policy binds neither the organization's management account nor service-linked roles, so if the flagship account is the management account A cannot stand on one; a resource control policy on the buckets is the nearer tool, and which account the flagship is sits on the list below. Who is on the named list, and who can assume each role on it, is decision 14. |
| +**A. Nobody else can reach a store bucket.** (Today the host role can give itself any store bucket's objects through a store user it makes: see decision 14 below. The binding this proposal asks for is exactly what that lacks.) Proposed direction: an organization policy (or, failing that, a permissions boundary required on every principal the platform creates plus an account rule) that denies S3 actions on the store bucket prefix to every principal except each bucket's own store user and a short list named by ARN. What that needs, beyond the sentence: a binding between each store identity and its one bucket, enforced (a store user under the path with a policy for another store's bucket must be impossible, not merely absent); coverage of the paths an identity policy does not show (resource policies and grants to other accounts, service-linked roles, access points); the enforcement configuration itself protected from the principals it constrains; and a way to verify the control is in force that is separate from this report, which cannot see it. An organization policy binds neither the organization's management account nor service-linked roles, and a resource control policy, the nearer tool for "nobody but X reaches these buckets", has the same two limits (it does not apply to resources in the management account and does not restrict service-linked roles); so if the flagship account is the management account neither can carry A, and which account the flagship is sits on the list below. Who is on the named list, and who can assume each role on it, is decision 14. |
| |
| **B. The bucket is the one this grant made.** Proposed direction: the grant mints a bucket name with a random part recorded in its grant block before any provider call, and refuses a bucket that already exists under that name. What that needs, beyond the sentence: randomness only makes an accidental collision and a guess unlikely; an administrator who can read the grant block, or the store user's policy written before the bucket, can create the name first, so creation provenance needs an enforceable restriction on who may create buckets under the prefix, not unpredictability; an explicit treatment of the provider answering an already-owned bucket's create as success (the us-east-1 case the provisioner already notes) and of a retry after the platform's own create; and crash recovery that tells the platform's own earlier create from someone else's. |
| |
| @@ -41,7 +41,7 @@ Neither is designed, and both change infrastructure or the grant walk, which nee |
| |
| ## Decision for Paul (item 14, reworded) |
| |
| -Which principals may reach tenant store buckets besides the tenant's own store user, and who can act as each. Today the host role (it creates and deletes store buckets; it cannot read objects) and any administrator role reach every store bucket. AWS charges transfer by bucket and does not say who moved the bytes: if staff download from a tenant's bucket, that transfer lands on the tenant's bill. |
| +Which principals may reach tenant store buckets besides the tenant's own store user, and who can act as each. Today the host role (it creates and deletes store buckets and holds no `s3:GetObject` of its own, but `infra/modules/region-host/main.tf` lets it create a store user and write that user's policy under the store boundary, which allows `s3:GetObject` on `carlos-store-*/*`, and mint the user's keys: so it, and whoever runs code on a host as it, can give itself any store bucket's objects through a user it makes, in three calls) and any administrator role reach every store bucket. AWS charges transfer by bucket and does not say who moved the bytes: if staff download from a tenant's bucket, that transfer lands on the tenant's bill. |
| |
| Proposed: only the deployment's host role and roles that need account administration to assume; named one by one, by ARN, with each role's trust policy part of the record; staff do not read tenant buckets, and if support ever requires it the platform credits the bytes. And approve the direction in A and B above, or say what should stand in their place. |
| |