amadan
per-environment object storage: teardown and metering
Purging an app now tears down EVERY store it has, not just its own. This is the other half of the purge guard slice 1 added: release.PurgeApp refuses the box-side auto-purge while any store under the app holds a grant or a provisioned block, so without this an app with an environment store could not be purged at all — by this door or any other. One lock per store, held across that store's read and its walk, the app's own last, so a run that stops part way still reads as "this app has a store", which is true and which re-entering resumes. An unreadable listing refuses the purge: "could not ask" is not "no store", and the sweep is irreversible. Destroying one store is the console door it already was, now aimed at an environment. There is no CLI verb, by ruling (spec §5): the ceremony here is owner, CSRF, the bucket name typed exactly and fresh passkey auth, and a bearer-token verb would have traded that away on the one action with no trash and no undo. The settings page grows a card per environment store, each with its own bucket, key age and confirm-and-destroy form — the app's own card renders byte-for-byte as before, which is what its existing test asserts. Metering sums every store of an app into the one existing usage key, so StorageDay's wire shape gains no dimension and a bill is the same arithmetic over more buckets. An unreadable environment record marks the day unswept rather than writing a zero into a create-only object that could never be corrected. Per-store attribution stays discussion #140's ask. The tests size two buckets differently (1000 + 2000) so a dropped store fails rather than passing on a coincidence, and drive the real grant doors so the provisioner genuinely holds two buckets and two IAM users for the purge to find. Spec: docs/superpowers/specs/2026-09-19-per-environment-store-design.md §§7-10 Discussion: amadan carlos/platform #159
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7 files changed,
+452
−75
docs/superpowers/specs/2026-09-19-per-environment-store-design.md+31 −10internal/console/apiscan_test.go+4 −5internal/console/appsettings.go+68 −7internal/console/economicspass.go+32 −13internal/console/ownership.go+84 −40internal/console/store_env_estate_test.go+187 −0internal/console/templates/app_settings.html+46 −0
diff --git a/docs/superpowers/specs/2026-09-19-per-environment-store-design.md b/docs/superpowers/specs/2026-09-19-per-environment-store-design.md| index a42db9724..a57845eab 100644 |
| --- a/docs/superpowers/specs/2026-09-19-per-environment-store-design.md |
| +++ b/docs/superpowers/specs/2026-09-19-per-environment-store-design.md |
| @@ -128,9 +128,19 @@ carlos store create --app a --environment e [--env-prefix P] |
| carlos store status --app a [--environment e] |
| carlos store grant --app a --environment e (operator, unchanged gates) |
| carlos store rotate --app a --environment e [--finish] |
| -carlos store destroy --app a --environment e (explicit teardown, §8) |
| ``` |
| +**(ruling, James 2026-09-21) There is no `carlos store destroy`.** An earlier |
| +draft of this section listed one. Destroying a store stays what it already |
| +was: a console action behind the heaviest ceremony in the platform — owner |
| +role, CSRF, the bucket name typed exactly, and fresh passkey auth. A CLI verb |
| +cannot ask for a passkey, so adding one would have quietly traded that |
| +ceremony for a bearer token on the one action with no trash and no undo. |
| + |
| +The console's destroy door takes the environment instead (§8), and the |
| +settings page grows one card per environment store, each with its own |
| +confirm-and-destroy form. |
| + |
| **(decision) `status` with no `--environment` lists every store the app has**, |
| the app's own first, then one line per environment: its bucket, its state, |
| and which instances read it. An app with only the default store prints what |
| @@ -210,17 +220,28 @@ the wrong bucket or claiming a clean result. |
| ## 10. Slices |
| -1. **The primitive.** `Environment` on the record, the key layout, the name |
| +1. **The primitive.** ✅ `Environment` on the record, the key layout, the name |
| builders and their length refusal, `writeEnv`'s delivery target, and the |
| - walks threaded through. Plus §7's purge guard and its test. No surface yet. |
| -2. **The doors.** `--environment` on create/status/grant/rotate/finish, the |
| + walks threaded through. Plus §7's purge guard and its test. |
| +2. **The doors.** ✅ `?environment=` on create/status/grant/rotate/finish, the |
| lock key, `storeReply` carrying the environment, and the refusals of §5. |
| -3. **The CLI and the docs.** The verbs above, `status` listing every store, |
| - and `docs/site/cli.md` — a new verb spelling is not done until that page |
| - names it. |
| -4. **Teardown.** `store destroy --environment`, the account-teardown path |
| - covering every store, and the console destroy door. |
| -5. **Metering.** §8's sum, and the "unused" line. |
| +3. **The CLI and the docs.** ✅ `--environment` on create/status/grant/rotate, |
| + `status` listing every store, and `docs/site/cli.md`. |
| +4. **Teardown.** ✅ The account-teardown path covering every store, and the |
| + console destroy door taking an environment — with one card and one |
| + confirm-and-destroy form per environment store on the settings page. No |
| + CLI verb, per §5's ruling. |
| +5. **Metering.** ✅ §8's sum, over the app's own store and every environment |
| + store, into the one existing usage key. |
| + |
| +**Not built, and named here so it is not mistaken for shipped: §8's "unused" |
| +line.** `store status` lists every environment that has a store, which is |
| +where an abandoned one becomes visible — but it does not say whether any |
| +INSTANCE is bound to that environment, which is what "unused" claims. Doing |
| +it properly means reading the app's instance records and joining on their |
| +environment binding, and a line that said "unused" from anything less would |
| +be a guess about a live bucket. The listing is the honest half; the join is |
| +follow-up work. |
| ## 11. Non-goals |
diff --git a/internal/console/apiscan_test.go b/internal/console/apiscan_test.go| index 0dffa612f..976aa3d1d 100644 |
| --- a/internal/console/apiscan_test.go |
| +++ b/internal/console/apiscan_test.go |
| @@ -15,13 +15,12 @@ import ( |
| func scanURL(fx *consoleFixture, app string) string { return storeURL(fx, app) + "/scan" } |
| // grantStore drives the real operator grant so the store is provisioned — the |
| -// state every scan verb but `request` needs to act on. |
| +// state every scan verb but `request` needs to act on. The app's OWN store: |
| +// grantStoreIn (store_env_estate_test.go) is this for any store of the app, |
| +// and carries the implementation, so the two cannot drift. |
| func grantStore(t *testing.T, fx *consoleFixture, token string) { |
| t.Helper() |
| - declareStore(t, fx, token) |
| - if status, out := postJSON(t, storeURL(fx, "titogo")+"/grant", map[string]any{}, token); status != http.StatusOK { |
| - t.Fatalf("grant = %d %v", status, out) |
| - } |
| + grantStoreIn(t, fx, token, "") |
| } |
| // scanArmed is storeArmed plus a granted store and a working engine. |
diff --git a/internal/console/appsettings.go b/internal/console/appsettings.go| index f6a9bef49..3cf6bf774 100644 |
| --- a/internal/console/appsettings.go |
| +++ b/internal/console/appsettings.go |
| @@ -106,6 +106,13 @@ type SettingsView struct { |
| // record, so the section remains present and does not claim there is no |
| // store during a transient failure. |
| StoreUnavailable bool |
| + // StoreEnvs is one card per ENVIRONMENT store (2026-09-19), oldest name |
| + // first, empty for the apps that have none — which is all of them until |
| + // someone asks for one. Each carries its own bucket, its own credential |
| + // and its own destroy form: this is the only screen where a store that |
| + // no instance reads is visible, and teardown is explicit by ruling, so |
| + // an unnoticed one keeps billing. |
| + StoreEnvs []*StoreCard |
| // Mail is the app's tenant email as this page sees it (issue #240) — |
| // object storage's sibling primitive, same declare -> provision -> |
| @@ -177,6 +184,10 @@ type GateRow struct { |
| // with, or when the record is a declaration nobody has acted on, which needs no |
| // backend at all because nothing outside the bucket exists yet. |
| type StoreCard struct { |
| + // Environment is the config environment this store belongs to, "" for |
| + // the app's own. The destroy form posts it back, so the door tears down |
| + // the store the card describes and no other. |
| + Environment string |
| Declared bool |
| EnvPrefix string |
| Granted bool |
| @@ -305,6 +316,7 @@ func (s *Server) assembleSettings(ctx context.Context, p *page, acct *Account) * |
| sv.CanRename = true |
| } |
| sv.Store = s.storeCard(ctx, acct, p.AppName) |
| + sv.StoreEnvs = s.storeEnvCards(ctx, acct, p.AppName) |
| sv.Mail = s.mailCard(ctx, acct, p.AppName) |
| sv.Features = s.featuresCard(ctx, acct, p.AppName) |
| sv.StoreUnavailable = sv.Store == nil |
| @@ -388,7 +400,47 @@ func (s *Server) storeCard(ctx context.Context, acct *Account, app string) *Stor |
| if rec == nil { |
| return &StoreCard{} |
| } |
| - card := &StoreCard{} |
| + return s.storeCardFor(rec, "") |
| +} |
| + |
| +// storeEnvCards is one card per environment store, in the order |
| +// StoreEnvironments returns them (sorted). nil on a read failure, matching |
| +// storeCard's own discipline: the page says nothing rather than claiming an |
| +// app has no environment stores when it could not find out. |
| +// |
| +// A single unreadable record does NOT blank the whole list — the others are |
| +// real and their buckets are real — so it is skipped, logged, and the rest |
| +// render. That is the opposite trade to storeCard's, and deliberately: |
| +// there, a nil is the difference between "no store" and "unknown"; here, |
| +// dropping every card because one is malformed would hide estate that is |
| +// perfectly legible. |
| +func (s *Server) storeEnvCards(ctx context.Context, acct *Account, app string) []*StoreCard { |
| + envs, err := objstore.StoreEnvironments(ctx, s.cfg.Releases, acct.Sqid, app) |
| + if err != nil { |
| + log.Printf("console: settings store environments %s/%s: %v", acct.Sqid, app, err) |
| + return nil |
| + } |
| + cards := make([]*StoreCard, 0, len(envs)) |
| + for _, env := range envs { |
| + rec, _, rerr := objstore.GetIn(ctx, s.cfg.Releases, acct.Sqid, app, env) |
| + if rerr != nil { |
| + log.Printf("console: settings store read %s/%s%s: %v", acct.Sqid, app, storeEnvSuffix(env), rerr) |
| + continue |
| + } |
| + if rec == nil { |
| + continue |
| + } |
| + cards = append(cards, s.storeCardFor(rec, env)) |
| + } |
| + return cards |
| +} |
| + |
| +// storeCardFor maps one record to the card the settings page renders. Shared |
| +// by both callers so the app's own store and an environment's can never |
| +// disagree about what a grant, a provisioning or a half-finished rotation |
| +// looks like. |
| +func (s *Server) storeCardFor(rec *objstore.Record, env string) *StoreCard { |
| + card := &StoreCard{Environment: env} |
| if rec.Declared != nil { |
| card.Declared = true |
| card.EnvPrefix = rec.Declared.EnvPrefix |
| @@ -987,14 +1039,23 @@ func (s *Server) handleStoreDestroy(w http.ResponseWriter, r *http.Request) { |
| // landing between reading it and tearing it down would make both stale — |
| // the shape objstore's deleteRecordIfUnchanged is guarding against from |
| // the other end (objstore/walk.go:560). |
| - defer s.storeWalks.lock(acct.Sqid, app, "")() |
| - rec, _, err := objstore.Get(ctx, s.cfg.Releases, acct.Sqid, app) |
| + // Which store: the app's own, or one belonging to a named config |
| + // environment. The form carries it; an unknown or malformed name reads |
| + // back as no record at all, which is the honest answer to "destroy the |
| + // store that isn't there" and the same one a tampered field gets. |
| + env := strings.TrimSpace(r.FormValue("environment")) |
| + defer s.storeWalks.lock(acct.Sqid, app, env)() |
| + rec, _, err := objstore.GetIn(ctx, s.cfg.Releases, acct.Sqid, app, env) |
| if err != nil { |
| - log.Printf("console: store destroy read %s/%s: %v", acct.Sqid, app, err) |
| + log.Printf("console: store destroy read %s/%s%s: %v", acct.Sqid, app, storeEnvSuffix(env), err) |
| fail("Could not read “" + app + "”'s object-store record — nothing was destroyed. Try again.") |
| return |
| } |
| if rec == nil { |
| + if env != "" { |
| + fail("“" + app + "” has no object store for the " + env + " environment, so there is nothing to destroy.") |
| + return |
| + } |
| fail("“" + app + "” has no object store, so there is nothing to destroy.") |
| return |
| } |
| @@ -1035,8 +1096,8 @@ func (s *Server) handleStoreDestroy(w http.ResponseWriter, r *http.Request) { |
| // button says "Destroy store", and the success flash below names only |
| // the object store. Email teardown belongs to ownership.go's |
| // handlePurgeApp, which destroys the whole app and says so. |
| - if err := objstore.Teardown(ctx, s.cfg.Releases, s.cfg.StoreProvisioner, s.cfg.StoreScanner, acct.Sqid, app); err != nil { |
| - log.Printf("console: store destroy %s/%s: %v", acct.Sqid, app, err) |
| + if err := objstore.TeardownIn(ctx, s.cfg.Releases, s.cfg.StoreProvisioner, s.cfg.StoreScanner, acct.Sqid, app, env); err != nil { |
| + log.Printf("console: store destroy %s/%s%s: %v", acct.Sqid, app, storeEnvSuffix(env), err) |
| if errors.Is(err, release.ErrCASConflict) { |
| fail("The store changed while you confirmed — nothing was destroyed; reload and try again.") |
| return |
| @@ -1051,7 +1112,7 @@ func (s *Server) handleStoreDestroy(w http.ResponseWriter, r *http.Request) { |
| flash = "Destroyed “" + app + "”'s object store: bucket " + want + ", its credentials and the " + |
| "variables they were delivered in are gone." |
| } |
| - s.audit(acct, app, addr, "store-destroy", "destroyed the object store ("+what+")") |
| + s.audit(acct, app, addr, "store-destroy", "destroyed the object store ("+what+")"+storeEnvSuffix(env)) |
| // Box-consumed: the delivered variables were UNSET in the app's config |
| // (objstore.Teardown), so every host still serving this app has an env file |
| // naming a bucket and a key that authenticate against nothing. Without the |
diff --git a/internal/console/economicspass.go b/internal/console/economicspass.go| index bfcb8a251..ab8f31261 100644 |
| --- a/internal/console/economicspass.go |
| +++ b/internal/console/economicspass.go |
| @@ -168,25 +168,44 @@ func (s *Server) storagePass(ctx context.Context, date string) error { |
| for _, ks := range sizes { |
| st.BucketBytes += ks.Size |
| } |
| - rec, _, gerr := objstore.Get(ctx, s.cfg.Releases, acct, app) |
| - if gerr != nil { |
| - // Absent is (nil, "", nil) — objstore.Get's own idiom — so a |
| - // non-nil error here is a REAL failure (transient bucket |
| - // read, malformed store.json), not "no store". Treating it |
| - // as "no store" would write StoreBytes:0/StoreUnswept:false |
| - // into a create-only object, indistinguishable from an app |
| - // that never had a store and never correctable afterward. |
| - log.Printf("console: storage pass: store record %s/%s: %v", acct, app, gerr) |
| + // EVERY store this app has: its own, and one per config |
| + // environment (2026-09-19). They sum into ONE figure under the |
| + // app's usage key, so StorageDay's wire shape gains no dimension |
| + // and an account's bill is the same arithmetic over more buckets. |
| + // Per-store attribution is discussion #140's ask, and deliberately |
| + // not made here. |
| + envs, eerr := objstore.StoreEnvironments(ctx, s.cfg.Releases, acct, app) |
| + if eerr != nil { |
| + // Same rule as an unreadable record below: "could not ask" is |
| + // not "no store", and this object is create-only. |
| + log.Printf("console: storage pass: store environments %s/%s: %v", acct, app, eerr) |
| st.StoreUnswept = true |
| - } else if rec != nil && rec.Provisioned != nil { |
| + } |
| + for _, env := range append([]string{""}, envs...) { |
| + rec, _, gerr := objstore.GetIn(ctx, s.cfg.Releases, acct, app, env) |
| + if gerr != nil { |
| + // Absent is (nil, "", nil) — objstore.GetIn's own idiom — |
| + // so a non-nil error here is a REAL failure (transient |
| + // bucket read, malformed store.json), not "no store". |
| + // Treating it as "no store" would write |
| + // StoreBytes:0/StoreUnswept:false into a create-only |
| + // object, indistinguishable from an app that never had a |
| + // store and never correctable afterward. |
| + log.Printf("console: storage pass: store record %s/%s%s: %v", acct, app, storeEnvSuffix(env), gerr) |
| + st.StoreUnswept = true |
| + continue |
| + } |
| + if rec == nil || rec.Provisioned == nil { |
| + continue |
| + } |
| st.StoreProvisioned = true |
| n, serr := s.sizeStore(ctx, rec.Provisioned) |
| if serr != nil { |
| - log.Printf("console: storage pass: store %s/%s: %v", acct, app, serr) |
| + log.Printf("console: storage pass: store %s/%s%s: %v", acct, app, storeEnvSuffix(env), serr) |
| st.StoreUnswept = true |
| - } else { |
| - st.StoreBytes = n |
| + continue |
| } |
| + st.StoreBytes += n |
| } |
| day.Apps[acct+"/"+app] = st |
| } |
diff --git a/internal/console/ownership.go b/internal/console/ownership.go| index 641ce50a2..b0b355d81 100644 |
| --- a/internal/console/ownership.go |
| +++ b/internal/console/ownership.go |
| @@ -397,49 +397,16 @@ func (s *Server) handlePurgeApp(w http.ResponseWriter, r *http.Request) { |
| fail("“" + app + "” has a provisioned email credential, and this console has no SES backend configured — it cannot be deleted here until email is torn down.") |
| return |
| } |
| - // A provisioned store is torn down BEFORE the prefix sweep — this door |
| - // is the only purge path with backend authority (objstore spec §9; |
| + // EVERY store this app has is torn down BEFORE the prefix sweep — this |
| + // door is the only purge path with backend authority (objstore spec §9; |
| // box-side auto-purge refuses via release.ErrStoreProvisioned). Failure |
| - // aborts the purge with the record intact: re-entering resumes. |
| - // Guard on Provisioned OR Grant (controller ruling, Task 5 review): a |
| - // grant block without a provisioned block is a crashed grant whose |
| - // backend estate is real. |
| - // Held across the read and the walk both: the estate check below is made |
| - // from this record, and a grant landing in between would leave the purge |
| - // sweeping a prefix whose bucket and IAM user were just re-created |
| - // (objstore/walk.go:560's window, from the other end). |
| - unlockStoreWalk := s.storeWalks.lock(acct.Sqid, app, "") |
| - defer unlockStoreWalk() |
| - if rec, _, rerr := objstore.Get(r.Context(), s.cfg.Releases, acct.Sqid, app); rerr == nil && rec != nil && (rec.Provisioned != nil || rec.Grant != nil) { |
| - if s.cfg.StoreProvisioner == nil { |
| - fail("“" + app + "” has a provisioned object store, and this console has no storage backend configured (CARLOS_STORE_BACKEND) — it cannot be deleted here until the store is torn down.") |
| - return |
| - } |
| - // The walk gets its own ceiling rather than the request's open-ended |
| - // context: a teardown is many round trips (EmptyBucket is paginated), |
| - // and it must not be able to hold this handler open indefinitely. |
| - tctx, cancel := context.WithTimeout(r.Context(), storeTeardownTimeout) |
| - defer cancel() |
| - if terr := objstore.Teardown(tctx, s.cfg.Releases, s.cfg.StoreProvisioner, s.cfg.StoreScanner, acct.Sqid, app); terr != nil { |
| - log.Printf("console: purge store teardown %s/%s: %v", acct.Sqid, app, terr) |
| - fail("Could not tear down “" + app + "”'s object store — nothing was deleted; try again.") |
| - return |
| - } |
| - // rec.Provisioned may be nil here (crashed-grant shape) — name the |
| - // bucket from Grant, which the guard condition proves non-nil when |
| - // Provisioned is. |
| - bucket := "" |
| - if rec.Provisioned != nil { |
| - bucket = rec.Provisioned.Bucket |
| - } else if rec.Grant != nil { |
| - bucket = rec.Grant.Bucket |
| - } |
| - s.audit(acct, app, addr, "store-destroy", "tore down the object store (bucket "+bucket+") during permanent deletion") |
| - } else if rerr != nil { |
| - log.Printf("console: purge store read %s/%s: %v", acct.Sqid, app, rerr) |
| - fail("Could not read “" + app + "”'s object-store record — nothing was deleted; try again.") |
| + // aborts the purge with the records intact: re-entering resumes. |
| + if msg := s.purgeAppStores(r.Context(), acct, app, addr); msg != "" { |
| + fail(msg) |
| return |
| } |
| + // (purgeAppStores is below.) |
| + // |
| // An email estate is torn down BEFORE the prefix sweep too, for the |
| // identical reason the store is above (I2, review follow-up): |
| // mail.Teardown is the ONLY place that releases this app's domain |
| @@ -714,3 +681,80 @@ func (s *Store) DeleteAccount(accountID int64, now time.Time) error { |
| } |
| return tx.Commit() |
| } |
| + |
| +// purgeAppStores tears down every store this app has — its own and one per |
| +// config environment — and returns the flash to show on failure, "" on |
| +// success. |
| +// |
| +// Since 2026-09-19 an app may have more than one store, and release.PurgeApp |
| +// refuses the box-side auto-purge while ANY of them holds a grant or a |
| +// provisioned block. So a purge that tore down only the app's own store would |
| +// not leave a partial cleanup behind: it would leave an app that can never be |
| +// purged at all, by this door or any other. |
| +// |
| +// One lock per store, held across that store's read AND its walk — the estate |
| +// check is made from the record, and a grant landing in between would leave |
| +// the sweep deleting a prefix whose bucket and IAM user were just re-created |
| +// (objstore/walk.go's window, from the other end). Taken in turn rather than |
| +// all at once: two stores of one app share no estate, so there is nothing to |
| +// serialise between them, and holding several locks to do nothing with them |
| +// is how a deadlock gets written later. |
| +func (s *Server) purgeAppStores(ctx context.Context, acct *Account, app, addr string) string { |
| + envs, err := objstore.StoreEnvironments(ctx, s.cfg.Releases, acct.Sqid, app) |
| + if err != nil { |
| + // Refusing on an unreadable listing is the same direction as |
| + // refusing on an unreadable record: "I could not ask" is not "no |
| + // store", and the sweep below is irreversible. |
| + log.Printf("console: purge store list %s/%s: %v", acct.Sqid, app, err) |
| + return "Could not list “" + app + "”'s object stores — nothing was deleted; try again." |
| + } |
| + // The app's own store LAST. It is the one whose grant block the purge |
| + // guard has always keyed on, so a run that stops part way leaves a state |
| + // that still reads as "this app has a store" — which is true, and which |
| + // re-entering resumes. |
| + for _, env := range append(envs, "") { |
| + if msg := s.purgeOneStore(ctx, acct, app, env, addr); msg != "" { |
| + return msg |
| + } |
| + } |
| + return "" |
| +} |
| + |
| +// purgeOneStore is purgeAppStores' body for a single store. Guard on |
| +// Provisioned OR Grant (controller ruling, Task 5 review): a grant block |
| +// without a provisioned block is a crashed grant whose backend estate is |
| +// real. |
| +func (s *Server) purgeOneStore(ctx context.Context, acct *Account, app, env, addr string) string { |
| + defer s.storeWalks.lock(acct.Sqid, app, env)() |
| + rec, _, rerr := objstore.GetIn(ctx, s.cfg.Releases, acct.Sqid, app, env) |
| + if rerr != nil { |
| + log.Printf("console: purge store read %s/%s%s: %v", acct.Sqid, app, storeEnvSuffix(env), rerr) |
| + return "Could not read “" + app + "”'s object-store record — nothing was deleted; try again." |
| + } |
| + if rec == nil || (rec.Provisioned == nil && rec.Grant == nil) { |
| + return "" |
| + } |
| + if s.cfg.StoreProvisioner == nil { |
| + return "“" + app + "” has a provisioned object store, and this console has no storage backend configured (CARLOS_STORE_BACKEND) — it cannot be deleted here until the store is torn down." |
| + } |
| + // The walk gets its own ceiling rather than the request's open-ended |
| + // context: a teardown is many round trips (EmptyBucket is paginated), and |
| + // it must not be able to hold this handler open indefinitely. |
| + tctx, cancel := context.WithTimeout(ctx, storeTeardownTimeout) |
| + defer cancel() |
| + if terr := objstore.TeardownIn(tctx, s.cfg.Releases, s.cfg.StoreProvisioner, s.cfg.StoreScanner, acct.Sqid, app, env); terr != nil { |
| + log.Printf("console: purge store teardown %s/%s%s: %v", acct.Sqid, app, storeEnvSuffix(env), terr) |
| + return "Could not tear down “" + app + "”'s object store — nothing was deleted; try again." |
| + } |
| + // rec.Provisioned may be nil here (crashed-grant shape) — name the bucket |
| + // from Grant, which the guard above proves non-nil when Provisioned is. |
| + bucket := "" |
| + if rec.Provisioned != nil { |
| + bucket = rec.Provisioned.Bucket |
| + } else if rec.Grant != nil { |
| + bucket = rec.Grant.Bucket |
| + } |
| + s.audit(acct, app, addr, "store-destroy", |
| + "tore down the object store (bucket "+bucket+")"+storeEnvSuffix(env)+" during permanent deletion") |
| + return "" |
| +} |
diff --git a/internal/console/store_env_estate_test.go b/internal/console/store_env_estate_test.go| new file mode 100644 |
| index 000000000..b3358f166 |
| --- /dev/null |
| +++ b/internal/console/store_env_estate_test.go |
| @@ -0,0 +1,187 @@ |
| +package console |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "net/http" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/objstore" |
| + "github.com/carlosframework/platform/internal/usage" |
| +) |
| + |
| +// An app may have several stores since 2026-09-19 — its own, and one per |
| +// config environment. Two places have to count them ALL: the metering pass, |
| +// which bills for them, and the purge path, which is the only door with the |
| +// authority to tear them down. |
| + |
| +func provisionedRecord(bucket string) *objstore.Record { |
| + now := time.Now().UTC() |
| + return &objstore.Record{ |
| + Declared: &objstore.Declared{EnvPrefix: "CARLOS_STORE", At: now, By: "member@example.com"}, |
| + Grant: &objstore.GrantBlock{Backend: "mem", Bucket: bucket, Source: objstore.GrantSourceOperator, At: now, By: "op"}, |
| + Provisioned: &objstore.Provisioned{Bucket: bucket, Region: "eu-west-1", IAMUser: "u-" + bucket, KeyID: "K", At: now}, |
| + } |
| +} |
| + |
| +func readStorageDay(t *testing.T, fx *consoleFixture, date string) usage.StorageDay { |
| + t.Helper() |
| + raw, _, err := fx.rel.Get(context.Background(), usage.StorageDayKey(date)) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var day usage.StorageDay |
| + if err := json.Unmarshal(raw, &day); err != nil { |
| + t.Fatal(err) |
| + } |
| + return day |
| +} |
| + |
| +// The bill is the sum over every store of the app. Sizes differ per bucket so |
| +// this fails if either store is dropped, rather than passing on a coincidence. |
| +func TestStoragePassSumsEveryStoreOfAnApp(t *testing.T) { |
| + ctx := context.Background() |
| + fx := newConsoleFixture(t) |
| + |
| + if _, err := objstore.Put(ctx, fx.rel, fixtureAcct, fixtureApp, provisionedRecord("app-bucket"), ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := objstore.PutIn(ctx, fx.rel, fixtureAcct, fixtureApp, "beta", provisionedRecord("beta-bucket"), ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + fx.srv.storeSizer = func(_ context.Context, p *objstore.Provisioned) (int64, error) { |
| + switch p.Bucket { |
| + case "app-bucket": |
| + return 1000, nil |
| + case "beta-bucket": |
| + return 2000, nil |
| + } |
| + t.Errorf("sized an unexpected bucket %q", p.Bucket) |
| + return 0, nil |
| + } |
| + |
| + if err := fx.srv.storagePass(ctx, "2026-09-19"); err != nil { |
| + t.Fatal(err) |
| + } |
| + app := readStorageDay(t, fx, "2026-09-19").Apps[fixtureAcct+"/"+fixtureApp] |
| + if app.StoreBytes != 3000 { |
| + t.Errorf("StoreBytes = %d, want 3000 (1000 app + 2000 beta)", app.StoreBytes) |
| + } |
| + if !app.StoreProvisioned { |
| + t.Error("StoreProvisioned = false, want true") |
| + } |
| + if app.StoreUnswept { |
| + t.Error("StoreUnswept = true, but every store was sized") |
| + } |
| +} |
| + |
| +// "Could not ask" is not "no bytes". The day object is create-only, so a |
| +// zero written here can never be corrected — an unreadable environment store |
| +// has to mark the day unswept, exactly as an unreadable app store does. |
| +func TestStoragePassMarksUnsweptOnAnUnreadableEnvironmentStore(t *testing.T) { |
| + ctx := context.Background() |
| + fx := newConsoleFixture(t) |
| + |
| + if _, err := objstore.Put(ctx, fx.rel, fixtureAcct, fixtureApp, provisionedRecord("app-bucket"), ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := fx.rel.Put(ctx, objstore.KeyIn(fixtureAcct, fixtureApp, "beta"), []byte("{not valid json"), ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + fx.srv.storeSizer = func(_ context.Context, p *objstore.Provisioned) (int64, error) { |
| + return 1000, nil |
| + } |
| + |
| + if err := fx.srv.storagePass(ctx, "2026-09-19"); err != nil { |
| + t.Fatal(err) |
| + } |
| + app := readStorageDay(t, fx, "2026-09-19").Apps[fixtureAcct+"/"+fixtureApp] |
| + if !app.StoreUnswept { |
| + t.Error("StoreUnswept = false, but one of this app's stores could not be read") |
| + } |
| + // The store it COULD read is still counted: a partial figure marked |
| + // unswept is honest, a dropped one is not. |
| + if app.StoreBytes != 1000 { |
| + t.Errorf("StoreBytes = %d, want the app store's 1000", app.StoreBytes) |
| + } |
| +} |
| + |
| +// grantStoreIn drives the real doors, so the provisioner genuinely holds a |
| +// bucket, an IAM user and a key for each store — the estate a purge has to |
| +// find. grantStore (apiscan_test.go) is this with no environment. |
| +func grantStoreIn(t *testing.T, fx *consoleFixture, token, env string) { |
| + t.Helper() |
| + declareURL, grantURL := storeURL(fx, "titogo"), storeURL(fx, "titogo")+"/grant" |
| + if env != "" { |
| + declareURL = storeEnvURL(fx, "titogo", env) |
| + grantURL += "?environment=" + env |
| + } |
| + if status, out := postJSON(t, declareURL, map[string]any{}, token); status != http.StatusOK { |
| + t.Fatalf("declare %q = %d %v", env, status, out) |
| + } |
| + if status, out := postJSON(t, grantURL, map[string]any{}, token); status != http.StatusOK { |
| + t.Fatalf("grant %q = %d %v", env, status, out) |
| + } |
| +} |
| + |
| +// Purging an app tears down EVERY store it has. release.PurgeApp refuses the |
| +// box-side auto-purge while any of them holds a grant or provisioned block, |
| +// so a purge that missed one would not leave a partial cleanup — it would |
| +// leave an app that can never be purged at all. |
| +func TestPurgeAppStoresTearsDownEveryStore(t *testing.T) { |
| + ctx := context.Background() |
| + fx, fake, token, _ := storeArmed(t, "code-purge-every-store") |
| + grantStoreIn(t, fx, token, "") |
| + grantStoreIn(t, fx, token, "beta") |
| + |
| + if len(fake.Buckets) != 2 || len(fake.Users) != 2 { |
| + t.Fatalf("fixture drift: buckets %v users %v, want one of each per store", fake.Buckets, fake.Users) |
| + } |
| + |
| + acct := &Account{ID: OpsAccountID, Sqid: fx.sqid} |
| + if msg := fx.srv.purgeAppStores(ctx, acct, "titogo", "paul@keymail.dev"); msg != "" { |
| + t.Fatalf("purgeAppStores refused: %s", msg) |
| + } |
| + if len(fake.Buckets) != 0 { |
| + t.Errorf("buckets survived the purge: %v", fake.Buckets) |
| + } |
| + if len(fake.Users) != 0 { |
| + t.Errorf("IAM users survived the purge: %v", fake.Users) |
| + } |
| + for _, env := range []string{"", "beta"} { |
| + rec, _, err := objstore.GetIn(ctx, fx.rel, fx.sqid, "titogo", env) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if rec != nil { |
| + t.Errorf("store record for %q survived the purge: %+v", env, rec) |
| + } |
| + } |
| +} |
| + |
| +// A console with no storage backend cannot tear an estate down, so it must |
| +// refuse rather than let the prefix sweep delete the only records naming a |
| +// live bucket and a live credential. |
| +func TestPurgeAppStoresRefusesWithNoProvisioner(t *testing.T) { |
| + ctx := context.Background() |
| + fx, fake, token, _ := storeArmed(t, "code-purge-no-backend") |
| + grantStoreIn(t, fx, token, "beta") |
| + fx.srv.cfg.StoreProvisioner = nil |
| + |
| + acct := &Account{ID: OpsAccountID, Sqid: fx.sqid} |
| + msg := fx.srv.purgeAppStores(ctx, acct, "titogo", "paul@keymail.dev") |
| + if msg == "" { |
| + t.Fatal("purgeAppStores allowed a purge it could not carry out") |
| + } |
| + if len(fake.Buckets) != 1 { |
| + t.Errorf("the refused purge touched the backend: %v", fake.Buckets) |
| + } |
| + rec, _, err := objstore.GetIn(ctx, fx.rel, fx.sqid, "titogo", "beta") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if rec == nil { |
| + t.Error("the refused purge deleted the record naming a live estate") |
| + } |
| +} |
diff --git a/internal/console/templates/app_settings.html b/internal/console/templates/app_settings.html| index d6a17e9cd..ea31a759f 100644 |
| --- a/internal/console/templates/app_settings.html |
| +++ b/internal/console/templates/app_settings.html |
| @@ -220,6 +220,52 @@ credentials — an operator of this deployment has to tear it down.</p> |
| and delivers its credentials into this app's config.</p> |
| {{end}} |
| {{end}}{{end}} |
| +{{with .Settings}}{{range .StoreEnvs}} |
| +<h2>Object storage — {{.Environment}} environment</h2> |
| +<p class="small">Its own bucket and its own credential, delivered into the |
| +<code>{{.Environment}}</code> config bundle — so only the instances bound to |
| +that environment can reach it.</p> |
| +<div class="tablewrap"> |
| +<table> |
| + <tr><th>Env prefix</th><th>Backend</th><th>Bucket</th><th>Access key</th><th>Key age</th></tr> |
| + <tr> |
| + <td>{{if .EnvPrefix}}<code>{{.EnvPrefix}}_*</code>{{else}}<span title="no declaration on the record">—</span>{{end}}</td> |
| + <td>{{if .Backend}}{{.Backend}}{{else}}<span title="not granted yet">—</span>{{end}}</td> |
| + <td>{{if .Bucket}}<code>{{.Bucket}}</code>{{else}}<span title="not granted yet">—</span>{{end}}</td> |
| + <td>{{if .KeyID}}<code>{{.KeyID}}</code>{{else}}<span title="no credentials delivered yet">—</span>{{end}}</td> |
| + <td>{{if .KeyAge}}{{.KeyAge}}{{else}}<span title="unknown">—</span>{{end}}</td> |
| + </tr> |
| +</table> |
| +</div> |
| +{{if and (not .Granted) (not .Provisioned)}}<p class="small">Declared and waiting: |
| +an operator of this deployment creates the bucket and delivers the credentials.</p>{{end}} |
| +{{if .PendingFinish}}<p class="small">A rotation is half-finished — two keys open |
| +this bucket until you retire the old one:</p> |
| +<pre><code>carlos store rotate --finish --app {{$.Detail.App}} --environment {{.Environment}}</code></pre>{{end}} |
| +{{if $.IsOwner}} |
| +<h3 class="danger-head">Destroy this store</h3> |
| +{{if .CanDestroy}} |
| +<p class="small">Destroying empties and deletes |
| +{{if .Bucket}}<code>{{.Bucket}}</code>{{else}}the store{{end}}, deletes the |
| +credentials that opened it, and removes the delivered variables from the |
| +<code>{{.Environment}}</code> bundle. The app's own store is not touched. There is |
| +no trash for a store: the objects in it are gone.</p> |
| +<form method="post" action="{{$.Base}}/apps/{{$.Detail.App}}/store/destroy" class="settings-row"> |
| + <input type="hidden" name="csrf" value="{{$.CSRF}}"> |
| + <input type="hidden" name="environment" value="{{.Environment}}"> |
| + <label class="visually-hidden" for="store-destroy-confirm-{{.Environment}}">Type {{if .Bucket}}the bucket name{{else}}the app's name{{end}} to confirm</label> |
| + <input type="text" id="store-destroy-confirm-{{.Environment}}" name="confirm" |
| + placeholder="type {{if .Bucket}}{{.Bucket}}{{else}}{{$.Detail.App}}{{end}} to confirm" autocomplete="off" required> |
| + <button type="submit" class="btn-sm btn-danger">Destroy store</button> |
| +</form> |
| +<p class="small">Confirms with your passkey.</p> |
| +{{else}} |
| +<p class="small">This console has no storage backend configured |
| +(<code>CARLOS_STORE_BACKEND</code>), so it cannot delete this bucket or its |
| +credentials — an operator of this deployment has to tear it down.</p> |
| +{{end}} |
| +{{end}} |
| +{{end}}{{end}} |
| {{end}} |
| {{if eq .SettingsSection "general"}} |
| <h2>Errors</h2> |