console, provision: the private-storage policy, and the create door that spends it (issue #412)
`Manifest.PrivateStorage` is the app's standing declaration that every instance it creates gets a config environment of its own and an object store bound to that environment. Written by the person-only enable door; spent by the create door, which an `instance`-role service credential already reaches. That split is the feature: a public signup path provisions a tenant's own bucket without ever holding the power to create one. A plain bool, not a pointer, unlike Record's Hibernate and StatusNamed. Their reason is convergence — a box re-reads them every pass, so absent has to differ from an explicit false — and it does not apply to a field read at one door at create time whose false IS the legacy behaviour. Pinned by a test that parses the bytes an older console wrote. The create door, in order: derive the environment from the host's first label (an explicit one in the request wins, and still gets a store — the policy fills a blank, it does not seize the axis), refuse the three names that cannot carry a store, check the ceiling, run the walk, and only then write the record. THE STORE IS PROVISIONED BEFORE THE RECORD. Record-first leaves an instance whose bundle has no coordinates: it wakes onto local disk, silently, with no verb to repair it. Store-first leaves a store no instance is bound to, which `store status` lists and which a retry of the same host reuses exactly, because the environment came from the host and GrantIn is idempotent. One window stays open and is logged in those words rather than papered over — the sibling-apex undo withdraws a record whose store survives it, and tearing that bucket down from a path nobody drove would make this the platform's first automatic destroyer of tenant data (#408 §3 rules it out). Not a second implementation of provisioning: the same lock, seal key, timeout, entitlement re-read and GrantSourceAccountApproval the store door uses. The env prefix is inherited from the app's own store, so an app reading JELLY_S3_* does not get a second vocabulary for one thing. CARLOS_STORE_MAX_PER_APP (default 100) caps environment stores per app, because a bucket plus an IAM user per instance is a bigger footprint than an instance and IAM's limits are per cloud account. Enforced on the MANUAL store door too — a ceiling only the automatic path respected would be one anyone could step over by hand — and re-using an environment that already has a store is never capped, or a retry would be refused by the ceiling it had just reached. The enable door refuses to declare a policy it knows cannot be honoured: no object-storage backend, or an unapproved account. That departs from its habit of writing whatever policy it is handed, and deliberately — a manifest that guarantees every future create fails is a trap that fires once per signup, in a door whose caller is a machine. Turning the policy OFF is never refused, so an account whose entitlement was later declined can get its create door working again without an operator. The IAM-ceiling guard's test is watched to fail: with the check removed the caller gets a 502 they cannot act on instead of a 400 naming the part to shorten. Its arithmetic (body 55, bucket 55 and legal, principal 68 and refused) is encoded so the guard cannot quietly become dead code — under any prefix longer than "carlos-store-" the bucket refusal always fires first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8 files changed,
+1007
−5
cmd/console/main.go+18 −0internal/console/apiinstances.go+126 −5internal/console/apiinstancesprivatestore_test.go+473 −0internal/console/apistore.go+11 −0internal/console/instanceprivatestore.go+301 −0internal/console/server.go+11 −0internal/provision/provision.go+20 −0internal/provision/provision_test.go+47 −0
diff --git a/cmd/console/main.go b/cmd/console/main.go| index f8716ab43..ae91ed32e 100644 |
| --- a/cmd/console/main.go |
| +++ b/cmd/console/main.go |
| @@ -293,6 +293,7 @@ func serve() { |
| ScanInterval: envDuration("CARLOS_STORE_SCAN_INTERVAL", 24*time.Hour), |
| ScanMaxSigAge: envDuration("CARLOS_CLAMD_MAX_SIG_AGE", 168*time.Hour), |
| StoreDeployment: storeDeployment, |
| + StoreMaxPerApp: envCount("CARLOS_STORE_MAX_PER_APP", 0), |
| StoreOperators: storeOperators(), |
| // CARLOS_ACME_DELEGATION_ZONE — the same variable the edge reads |
| // (internal/edge/wildcard.go's newWildcardFromEnv) for the same |
| @@ -1005,6 +1006,23 @@ func envDuration(k string, def time.Duration) time.Duration { |
| return d |
| } |
| +// envCount reads a positive count. Zero from an UNSET variable means "the |
| +// console's own default" (the caller supplies def = 0 and the server side |
| +// substitutes); an explicitly set zero or negative is fatal, because a |
| +// deployment that meant to cap something and typed it wrong should be told |
| +// rather than quietly given the default it was trying to change. |
| +func envCount(k string, def int) int { |
| + v := strings.TrimSpace(os.Getenv(k)) |
| + if v == "" { |
| + return def |
| + } |
| + n, err := strconv.Atoi(v) |
| + if err != nil || n <= 0 { |
| + log.Fatalf("console: %s %q must be a positive whole number", k, v) |
| + } |
| + return n |
| +} |
| + |
| // envBytes reads a plain byte count. Same fatal-on-garbage posture. |
| func envBytes(k string, def int64) int64 { |
| v := strings.TrimSpace(os.Getenv(k)) |
diff --git a/internal/console/apiinstances.go b/internal/console/apiinstances.go| index 1256c0024..48e87f5d4 100644 |
| --- a/internal/console/apiinstances.go |
| +++ b/internal/console/apiinstances.go |
| @@ -536,6 +536,12 @@ type apiRecordView struct { |
| provision.Record |
| HiddenGate string `json:"gate,omitempty"` |
| Gated bool `json:"gated"` |
| + // PrivateStoreBucket is the bucket the create door provisioned for this |
| + // instance under the app's private-storage policy (issue #412), empty on |
| + // every other record and every other door. Not part of the record: it is |
| + // a fact about what this one request did, and the record's own |
| + // Environment is what names the store afterwards. |
| + PrivateStoreBucket string `json:"private_store_bucket,omitempty"` |
| } |
| func recordView(rec provision.Record) apiRecordView { |
| @@ -826,6 +832,33 @@ func (s *Server) handleAPIInstanceCreate(w http.ResponseWriter, r *http.Request) |
| if rec.Kind == "static" && platformNamespace && rec.Pool == "" { |
| rec.Steering = "latency" |
| } |
| + // The app's private-storage policy (issue #412, spec §6): every instance |
| + // gets a config environment of its own, named for the host's first |
| + // label, with an object store bound to it. |
| + // |
| + // An EXPLICIT environment in the request wins, and the policy still |
| + // applies to it. The policy fills a blank; it does not seize the axis. |
| + // What it promises is that this record's bundle carries the store |
| + // coordinates — not that the bundle has any particular name — so |
| + // honouring the caller's own name keeps `carlos canary` working on such |
| + // an app, and two instances naming one environment share its store |
| + // exactly as they already share its config (#408 §5: sharing is |
| + // visible, not refused). |
| + // |
| + // Stamped BEFORE ValidateShape so the shape rail sees the same |
| + // environment the box will, and before the ceiling and the walk below so |
| + // neither can be asked about a record that was never legal. |
| + if m.PrivateStorage { |
| + env, derived := rec.Environment, false |
| + if env == "" { |
| + env, derived = rec.Slug, true |
| + } |
| + if ref := s.instanceStoreNames(acct, app, env, derived); ref != nil { |
| + jsonError(w, ref.Status, ref.Msg) |
| + return |
| + } |
| + rec.Environment = env |
| + } |
| // The shape rails, at the door — kind vocabulary, one upstream shape per |
| // record, addr syntax, and the combinations that would wedge a host (a |
| // static route with an upstream, a hibernating addr-backed one). The |
| @@ -837,6 +870,47 @@ func (s *Server) handleAPIInstanceCreate(w http.ResponseWriter, r *http.Request) |
| jsonError(w, http.StatusBadRequest, err.Error()) |
| return |
| } |
| + // THE STORE IS PROVISIONED BEFORE THE RECORD IS WRITTEN, and a failure |
| + // here means no record at all (issue #412, spec §7). This ordering is |
| + // the load-bearing decision of the feature, so it is argued here rather |
| + // than assumed: |
| + // |
| + // Record first, and a failed walk leaves an instance whose bundle has no |
| + // store coordinates. It wakes, finds none, and runs on local disk — the |
| + // exact outcome the policy was declared to prevent, arrived at silently, |
| + // with no verb to repair it (there is no `instances create --resume`). |
| + // |
| + // Store first, and a failed record write leaves a store for an |
| + // environment no instance is bound to. That is visible — `store status` |
| + // lists every store of the app — and, because the environment is DERIVED |
| + // FROM THE HOST, a retry of the same host reuses it exactly: GrantIn is |
| + // idempotent, so the second attempt short-circuits and costs nothing. |
| + // |
| + // So: no instance record is ever created under this policy without its |
| + // store already provisioned and delivered. |
| + privateBucket := "" |
| + if m.PrivateStorage { |
| + if ref := s.storeCeiling(ctx, acct, app, rec.Environment); ref != nil { |
| + jsonError(w, ref.Status, ref.Msg) |
| + return |
| + } |
| + // r.Context(), not ctx: the walk takes storeGrantTimeout, which is |
| + // longer than this handler's own deadline. |
| + storeRec, ok := s.instanceStoreProvision(r.Context(), w, acct, app, rec.Environment, principal) |
| + if !ok { |
| + return |
| + } |
| + privateBucket = storeBucketName(storeRec) |
| + // And a fresh deadline for what is left. The walk can have consumed |
| + // every second of the handler's original budget, and the record |
| + // write below inheriting a spent context would fail immediately |
| + // after the bucket had already been paid for — the one shape of |
| + // failure this whole ordering exists to avoid. |
| + cancel() |
| + var recancel context.CancelFunc |
| + ctx, recancel = context.WithTimeout(r.Context(), instanceStoreTimeout) |
| + defer recancel() |
| + } |
| if err := provision.CreateRecord(ctx, s.cfg.Releases, rec); err != nil { |
| if errors.Is(err, release.ErrCASConflict) { |
| // A lost race against a second signup for the same host, or a |
| @@ -858,6 +932,15 @@ func (s *Server) handleAPIInstanceCreate(w http.ResponseWriter, r *http.Request) |
| if derr := provision.DeleteRecord(ctx, s.cfg.Releases, acct.Sqid, app, host); derr != nil { |
| log.Printf("console: api instances create %s/%s/%s: %s was claimed mid-create and UNDO FAILED: %v", acct.Sqid, app, host, siblingLabel, derr) |
| } |
| + if privateBucket != "" { |
| + // The one window where this feature's ordering leaves |
| + // residue, logged in those words rather than papered over: |
| + // tearing the bucket down from a path nobody drove would |
| + // make this the platform's first automatic destroyer of |
| + // tenant data, which the per-environment store spec rules |
| + // out (#408 §3). It is listed by `carlos store status`. |
| + log.Printf("console: api instances create %s/%s/%s: %s was claimed mid-create; the record was withdrawn and its store %s WAS LEFT BEHIND", acct.Sqid, app, host, siblingLabel, privateBucket) |
| + } |
| jsonError(w, http.StatusConflict, |
| host+" is app "+siblingLabel+"'s platform apex — it was created while this instance was being written; the instance was not kept") |
| return |
| @@ -877,7 +960,9 @@ func (s *Server) handleAPIInstanceCreate(w http.ResponseWriter, r *http.Request) |
| // a box-consumed write (carlos-deploy §3): announce it, and the instance |
| // is serving seconds after signup instead of on the hour. |
| s.bumpSerial(ctx) |
| - writeJSON(w, http.StatusCreated, recordView(rec)) |
| + out := recordView(rec) |
| + out.PrivateStoreBucket = privateBucket |
| + writeJSON(w, http.StatusCreated, out) |
| } |
| // instanceChannel answers "which channel is this record born on" for a |
| @@ -1472,9 +1557,15 @@ func (s *Server) handleAPIInstanceEnable(w http.ResponseWriter, r *http.Request) |
| // different requests on the wire (the apiConfigApply precedent): an |
| // omitted field preserves what the manifest already carries, an explicit |
| // value replaces it — including seed's explicit [] meaning "clear". |
| + // |
| + // PrivateStorage joined them on 2026-09-22 (issue #412) as a pointer for |
| + // exactly the same reason: an app that already gives its instances their |
| + // own storage must not lose the policy to a `carlos instances enable |
| + // -channel beta` that never mentioned it. |
| var req struct { |
| - Channel *string `json:"channel"` |
| - Seed *[]string `json:"seed"` |
| + Channel *string `json:"channel"` |
| + Seed *[]string `json:"seed"` |
| + PrivateStorage *bool `json:"private_storage"` |
| } |
| if !readJSON(w, r, &req) { |
| return |
| @@ -1547,6 +1638,21 @@ func (s *Server) handleAPIInstanceEnable(w http.ResponseWriter, r *http.Request) |
| if req.Seed != nil { |
| m.Seed = *req.Seed |
| } |
| + // Turning the policy ON is checked against what this deployment and this |
| + // account can actually do; turning it OFF never is (spec §5). The |
| + // asymmetry is deliberate: a policy that cannot be honoured fails once |
| + // per signup in a door whose caller is a machine, so the person |
| + // declaring it is told now — but an account whose entitlement was later |
| + // declined must be able to get its create door working again without |
| + // waiting on an operator. |
| + if req.PrivateStorage != nil && *req.PrivateStorage && !m.PrivateStorage { |
| + if !s.instancePrivateStorageAllowed(ctx, w, acct) { |
| + return |
| + } |
| + } |
| + if req.PrivateStorage != nil { |
| + m.PrivateStorage = *req.PrivateStorage |
| + } |
| if err := provision.PutManifestIfMatch(ctx, s.cfg.Releases, acct.Sqid, app, *m, etag); err != nil { |
| if errors.Is(err, release.ErrCASConflict) { |
| jsonError(w, http.StatusConflict, |
| @@ -1557,16 +1663,31 @@ func (s *Server) handleAPIInstanceEnable(w http.ResponseWriter, r *http.Request) |
| jsonError(w, http.StatusBadGateway, "could not enable instances — try again") |
| return |
| } |
| - s.audit(acct, app, addr, "instance-enable", "enabled instances under "+domain+" (cli)") |
| + s.audit(acct, app, addr, "instance-enable", "enabled instances under "+domain+privateStorageAuditSuffix(req.PrivateStorage)+" (cli)") |
| // The manifest names the channel every minted route follows, so a change |
| // here changes what boxes converge onto — reconcile reads it each pass. |
| s.bumpSerial(ctx) |
| writeJSON(w, http.StatusOK, map[string]any{ |
| "app": app, "domain": m.Domain, "pool": m.Pool, "channel": m.Channel, |
| - "regions": m.Regions, "seed": m.Seed, |
| + "regions": m.Regions, "seed": m.Seed, "private_storage": m.PrivateStorage, |
| }) |
| } |
| +// privateStorageAuditSuffix names the policy in the audit line only when |
| +// this request SPOKE to it. An enable that changed a channel says nothing |
| +// about storage, and a row that restated the standing policy every time |
| +// would make the one request that changed it impossible to find. |
| +func privateStorageAuditSuffix(req *bool) string { |
| + switch { |
| + case req == nil: |
| + return "" |
| + case *req: |
| + return ", every instance with storage of its own" |
| + default: |
| + return ", instances sharing the app's storage" |
| + } |
| +} |
| + |
| // instanceRegionAllowed applies the manifest's region allow-list. An empty |
| // list means any shape-valid region (spec §3.1) — including the empty, |
| // unconstrained one, which is what a single-region deployment uses. |
diff --git a/internal/console/apiinstancesprivatestore_test.go b/internal/console/apiinstancesprivatestore_test.go| new file mode 100644 |
| index 000000000..33c08936e |
| --- /dev/null |
| +++ b/internal/console/apiinstancesprivatestore_test.go |
| @@ -0,0 +1,473 @@ |
| +package console |
| + |
| +// The app-declared private-storage policy (issue #412). What these pin, in |
| +// the order they would hurt if lost: |
| +// |
| +// - the create door provisions the store BEFORE it writes the record, so |
| +// a failed walk leaves no instance at all — never an instance that wakes |
| +// onto local disk because its bundle had no coordinates; |
| +// - the environment is the host's first label, and the coordinates land in |
| +// THAT bundle, not the app's default one; |
| +// - an app without the policy is byte-for-byte unchanged; |
| +// - the ceiling holds on both doors, and re-using an existing store is not |
| +// capped; |
| +// - declaring the policy is a person's act, refused in the two states |
| +// where it could only ever be a trap. |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "net/http" |
| + "os" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/appconfig" |
| + "github.com/carlosframework/platform/internal/objstore" |
| + "github.com/carlosframework/platform/internal/provision" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// privateStorageFixture is instanceFixture with the store plane armed: a |
| +// backend, a sealing key, and an approved account entitlement — the three |
| +// things the policy spends, and the standing approval #408 §6 says is the |
| +// only one needed. |
| +func privateStorageFixture(t *testing.T, code string) (*consoleFixture, *objstore.MemProvisioner, string, string) { |
| + t.Helper() |
| + fx, token := instanceFixture(t, code) |
| + pub, priv, _ := sealedFor(t, "irrelevant") |
| + fake := objstore.NewMemProvisioner("eu-west-1") |
| + fx.srv.cfg.SecretsPub = pub |
| + fx.srv.cfg.StoreProvisioner = fake |
| + fx.srv.cfg.StoreDeployment = objstore.Deployment{Backend: "mem", Region: "eu-west-1", BucketPrefix: "carlos-"} |
| + approveStoreAccount(t, fx, "operator@keymail.dev") |
| + return fx, fake, token, priv |
| +} |
| + |
| +// privateManifest is titoManifest carrying the policy. The domain is the |
| +// app's own (not the platform namespace), which keeps the sibling-apex |
| +// branch out of these tests — it has its own coverage. |
| +func privateManifest() provision.Manifest { |
| + m := titoManifest() |
| + m.PrivateStorage = true |
| + return m |
| +} |
| + |
| +func createInstance(t *testing.T, fx *consoleFixture, token, body string) (int, string) { |
| + t.Helper() |
| + status, _, out := rawReq(t, http.MethodPost, instancesURL(fx, "bab", "titogo"), token, body) |
| + return status, out |
| +} |
| + |
| +func envStore(t *testing.T, fx *consoleFixture, env string) *objstore.Record { |
| + t.Helper() |
| + rec, _, err := objstore.GetIn(context.Background(), fx.rel, release.OpsAccount, "titogo", env) |
| + if err != nil { |
| + t.Fatalf("read %s store: %v", env, err) |
| + } |
| + return rec |
| +} |
| + |
| +func envVars(t *testing.T, fx *consoleFixture, env string) map[string]appconfig.Var { |
| + t.Helper() |
| + m, err := appconfig.LoadEnvironment(context.Background(), fx.rel, release.OpsAccount, "titogo", env) |
| + if err != nil { |
| + t.Fatalf("read %s bundle: %v", env, err) |
| + } |
| + return m.Vars |
| +} |
| + |
| +// The whole chain, as a signup path drives it: one instance create, and the |
| +// tenant has a bucket of its own whose coordinates are already in the bundle |
| +// the record binds to. No `store create`, no `store provision`, no operator. |
| +func TestInstanceCreateProvisionsPrivateStore(t *testing.T) { |
| + fx, fake, token, priv := privateStorageFixture(t, "code-priv-chain") |
| + enableInstances(t, fx, "titogo", privateManifest()) |
| + |
| + status, out := createInstance(t, fx, token, `{"host":"acme.go.tito.io","region":"ie"}`) |
| + if status != http.StatusCreated { |
| + t.Fatalf("create = %d %s, want 201", status, out) |
| + } |
| + var echoed struct { |
| + Environment string `json:"environment"` |
| + PrivateStoreBucket string `json:"private_store_bucket"` |
| + } |
| + if err := json.Unmarshal([]byte(out), &echoed); err != nil { |
| + t.Fatalf("decode %s: %v", out, err) |
| + } |
| + // The environment is the host's FIRST LABEL, which is the one name |
| + // predictable from the host (spec §6). |
| + if echoed.Environment != "acme" { |
| + t.Fatalf("echoed environment = %q, want acme", echoed.Environment) |
| + } |
| + if echoed.PrivateStoreBucket != "carlos-bab-titogo-acme" { |
| + t.Fatalf("echoed bucket = %q, want carlos-bab-titogo-acme", echoed.PrivateStoreBucket) |
| + } |
| + // The record a box reconciles from carries the binding. |
| + stored, err := provision.GetRecord(context.Background(), fx.rel, "bab", "titogo", "acme.go.tito.io") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if stored.Environment != "acme" { |
| + t.Fatalf("stored environment = %q, want acme", stored.Environment) |
| + } |
| + // The store is real, provisioned, and its own. |
| + rec := envStore(t, fx, "acme") |
| + if rec == nil || rec.Provisioned == nil { |
| + t.Fatalf("acme store = %+v, want a provisioned one", rec) |
| + } |
| + if rec.Provisioned.Bucket != "carlos-bab-titogo-acme" || rec.Provisioned.IAMUser != "carlos-store-bab-titogo-acme" { |
| + t.Fatalf("provisioned = %+v, want the account/app/environment triple", rec.Provisioned) |
| + } |
| + // The grant's authority is the ACCOUNT entitlement, not an operator — |
| + // which is the whole reason no new power was granted to anyone. |
| + if rec.Grant == nil || rec.Grant.Source != objstore.GrantSourceAccountApproval { |
| + t.Fatalf("grant = %+v, want source %q", rec.Grant, objstore.GrantSourceAccountApproval) |
| + } |
| + if _, ok := fake.Buckets["carlos-bab-titogo-acme"]; !ok { |
| + t.Fatalf("backend buckets = %v, want the instance's own", fake.Buckets) |
| + } |
| + // Delivery goes to the INSTANCE's bundle. A credential in the app's |
| + // default bundle would be read by every other instance — the isolation |
| + // failing in exactly the direction that matters. |
| + vars := envVars(t, fx, "acme") |
| + if v, ok := vars["CARLOS_STORE_BUCKET"]; !ok || v.Value != "carlos-bab-titogo-acme" { |
| + t.Fatalf("acme bundle CARLOS_STORE_BUCKET = %+v, want the instance's bucket", v) |
| + } |
| + secret, ok := vars["CARLOS_STORE_SECRET_KEY"] |
| + if !ok || secret.Kind != "secret" { |
| + t.Fatalf("acme bundle CARLOS_STORE_SECRET_KEY = %+v, want a sealed secret", secret) |
| + } |
| + if openSecret(t, priv, secret) == "" { |
| + t.Fatal("the delivered secret did not open") |
| + } |
| + if got := appVars(t, fx, "titogo"); len(got) != 0 { |
| + t.Fatalf("the app's default bundle = %v, want nothing — a shared credential is the isolation failing", got) |
| + } |
| +} |
| + |
| +// An app without the policy is what it was before this branch: a record, no |
| +// store, no bundle, and no backend call. This is the test that fails if the |
| +// policy is ever read as on-by-default. |
| +func TestInstanceCreateWithoutPolicyTouchesNoStore(t *testing.T) { |
| + fx, fake, token, _ := privateStorageFixture(t, "code-priv-off") |
| + enableInstances(t, fx, "titogo", titoManifest()) |
| + |
| + status, out := createInstance(t, fx, token, `{"host":"acme.go.tito.io","region":"ie"}`) |
| + if status != http.StatusCreated { |
| + t.Fatalf("create = %d %s, want 201", status, out) |
| + } |
| + stored, err := provision.GetRecord(context.Background(), fx.rel, "bab", "titogo", "acme.go.tito.io") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if stored.Environment != "" { |
| + t.Fatalf("stored environment = %q, want empty — no policy, no derived binding", stored.Environment) |
| + } |
| + if rec := envStore(t, fx, "acme"); rec != nil { |
| + t.Fatalf("a store was created without the policy: %+v", rec) |
| + } |
| + untouched(t, fake, "a create with no private-storage policy") |
| +} |
| + |
| +// The policy fills a blank; it does not seize the axis. A caller who names |
| +// an environment gets a store for THAT one — which is still the invariant, |
| +// since what is promised is that the record's bundle carries the |
| +// coordinates, not that the bundle has any particular name. |
| +func TestInstanceCreateHonoursAnExplicitEnvironment(t *testing.T) { |
| + fx, _, token, _ := privateStorageFixture(t, "code-priv-explicit") |
| + enableInstances(t, fx, "titogo", privateManifest()) |
| + |
| + status, out := createInstance(t, fx, token, |
| + `{"host":"acme.go.tito.io","region":"ie","environment":"shared-eu"}`) |
| + if status != http.StatusCreated { |
| + t.Fatalf("create = %d %s, want 201", status, out) |
| + } |
| + stored, err := provision.GetRecord(context.Background(), fx.rel, "bab", "titogo", "acme.go.tito.io") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if stored.Environment != "shared-eu" { |
| + t.Fatalf("stored environment = %q, want the one the caller named", stored.Environment) |
| + } |
| + if rec := envStore(t, fx, "shared-eu"); rec == nil || rec.Provisioned == nil { |
| + t.Fatalf("shared-eu store = %+v, want a provisioned one", rec) |
| + } |
| + if rec := envStore(t, fx, "acme"); rec != nil { |
| + t.Fatalf("a store was also made for the derived name: %+v", rec) |
| + } |
| +} |
| + |
| +// A DNS label is not an environment name, and the create door is where that |
| +// has to be said: labels run to 63 characters, environment names to 32, and |
| +// "default" is a legal label that names the app's own bundle. Every refusal |
| +// leaves nothing behind. |
| +func TestInstanceCreateRefusesALabelThatCannotNameAStore(t *testing.T) { |
| + fx, fake, token, _ := privateStorageFixture(t, "code-priv-badlabel") |
| + enableInstances(t, fx, "titogo", privateManifest()) |
| + |
| + for _, label := range []string{ |
| + "default", |
| + "this-label-is-far-too-long-to-be-an-environment-name", |
| + } { |
| + host := label + ".go.tito.io" |
| + status, out := createInstance(t, fx, token, fmt.Sprintf(`{"host":%q,"region":"ie"}`, host)) |
| + if status != http.StatusBadRequest { |
| + t.Errorf("create at %s = %d %s, want 400", host, status, out) |
| + continue |
| + } |
| + if _, err := provision.GetRecord(context.Background(), fx.rel, "bab", "titogo", host); !errors.Is(err, os.ErrNotExist) { |
| + t.Errorf("a refused create at %s left a record (err %v)", host, err) |
| + } |
| + } |
| + untouched(t, fake, "a create whose label cannot name a store") |
| +} |
| + |
| +// An EXPLICIT environment gets the same rails, and the same 400. Without |
| +// this the refusal comes from inside the walk instead — a 502 the caller |
| +// cannot act on, over a name they could have shortened. |
| +func TestInstanceCreateRefusesAnUnnameableExplicitEnvironment(t *testing.T) { |
| + fx, fake, token, _ := privateStorageFixture(t, "code-priv-badenv") |
| + enableInstances(t, fx, "titogo", privateManifest()) |
| + // 32 characters — a legal environment name, and an IAM principal of |
| + // "carlos-store-bab-titogo-" + 32 = 56, under the ceiling. So this is |
| + // the bucket half: "carlos-" + "bab-titogo-" + 32 = 50. Both fit, so |
| + // the case that must refuse is the one the config plane itself refuses. |
| + status, out := createInstance(t, fx, token, |
| + `{"host":"acme.go.tito.io","region":"ie","environment":"UPPER"}`) |
| + if status != http.StatusBadRequest { |
| + t.Fatalf("create with an unusable environment = %d %s, want 400", status, out) |
| + } |
| + if _, err := provision.GetRecord(context.Background(), fx.rel, "bab", "titogo", "acme.go.tito.io"); !errors.Is(err, os.ErrNotExist) { |
| + t.Errorf("a refused create left a record (err %v)", err) |
| + } |
| + untouched(t, fake, "a create whose explicit environment cannot name a store") |
| +} |
| + |
| +// The IAM ceiling, and the arithmetic that makes it reachable at all. Both |
| +// names carry the same body: the bucket is <prefix><body> against S3's 63, |
| +// the principal is "carlos-store-" + <body> against IAM's 64. So under any |
| +// prefix longer than "carlos-store-" the BUCKET refusal always fires first, |
| +// and the IAM guard is only ever reached on a short prefix with a long app. |
| +// Encoded here so the guard cannot quietly become dead code. |
| +// |
| +// body = "bab" + "-" + app(25) + "-" + env(25) = 55. |
| +// bucket = "" + 55 = 55, legal. principal = 13 + 55 = 68, one refusal over. |
| +func TestInstanceCreateRefusesAnUnnameableIAMPrincipal(t *testing.T) { |
| + const app = "a-rather-long-application" // 25 |
| + label := strings.Repeat("b", 25) |
| + |
| + fx, token := instanceFixture(t, "code-priv-iamceiling") |
| + if _, err := fx.st.CreateAppIn(OpsAccountID, app, "paul@keymail.dev", time.Now()); err != nil { |
| + t.Fatal(err) |
| + } |
| + pub, _, _ := sealedFor(t, "irrelevant") |
| + fake := objstore.NewMemProvisioner("eu-west-1") |
| + fx.srv.cfg.SecretsPub = pub |
| + fx.srv.cfg.StoreProvisioner = fake |
| + fx.srv.cfg.StoreDeployment = objstore.Deployment{Backend: "mem", Region: "eu-west-1"} |
| + approveStoreAccount(t, fx, "operator@keymail.dev") |
| + enableInstances(t, fx, app, provision.Manifest{ |
| + Schema: 1, App: app, Domain: "go.tito.io", Channel: "stable", PrivateStorage: true}) |
| + |
| + host := label + ".go.tito.io" |
| + status, _, out := rawReq(t, http.MethodPost, instancesURL(fx, "bab", app), token, |
| + fmt.Sprintf(`{"host":%q}`, host)) |
| + if status != http.StatusBadRequest { |
| + t.Fatalf("create = %d %s, want 400", status, out) |
| + } |
| + if !strings.Contains(out, "64") { |
| + t.Errorf("refusal %q does not name AWS's ceiling", out) |
| + } |
| + if _, err := provision.GetRecord(context.Background(), fx.rel, "bab", app, host); !errors.Is(err, os.ErrNotExist) { |
| + t.Errorf("a refused create left a record (err %v)", err) |
| + } |
| + untouched(t, fake, "a create whose principal cannot be named") |
| +} |
| + |
| +// A ceiling only the automatic path respected would be a ceiling anyone |
| +// could step over by hand. |
| +func TestStoreCreateRefusedAtTheCeiling(t *testing.T) { |
| + fx, token := storeFixture(t, "code-store-ceiling") |
| + fx.srv.cfg.StoreMaxPerApp = 1 |
| + |
| + if status, body := postRaw(t, storeEnvURL(fx, "titogo", "one"), map[string]any{}, token); status != http.StatusOK { |
| + t.Fatalf("declare one = %d %s", status, body) |
| + } |
| + status, body := postRaw(t, storeEnvURL(fx, "titogo", "two"), map[string]any{}, token) |
| + if status != http.StatusConflict { |
| + t.Fatalf("declare past the cap = %d %s, want 409", status, body) |
| + } |
| + // Re-declaring the one that exists is not a new footprint, so it stands. |
| + if status, body := postRaw(t, storeEnvURL(fx, "titogo", "one"), map[string]any{}, token); status != http.StatusOK { |
| + t.Fatalf("re-declare one = %d %s, want 200", status, body) |
| + } |
| + // And the app's OWN store is not one of the capped collection. |
| + if status, body := postRaw(t, storeURL(fx, "titogo"), map[string]any{}, token); status != http.StatusOK { |
| + t.Fatalf("declare the app's own store = %d %s, want 200", status, body) |
| + } |
| +} |
| + |
| +// An app that already declared a prefix reads those names in its code. An |
| +// instance store delivering CARLOS_STORE_BUCKET into the same app would be a |
| +// second vocabulary for one thing. |
| +func TestInstanceStoreInheritsTheAppsEnvPrefix(t *testing.T) { |
| + fx, _, token, _ := privateStorageFixture(t, "code-priv-prefix") |
| + enableInstances(t, fx, "titogo", privateManifest()) |
| + if status, out := postRaw(t, storeURL(fx, "titogo"), map[string]any{"env_prefix": "JELLY_S3"}, token); status != http.StatusOK { |
| + t.Fatalf("declare the app store = %d %v", status, out) |
| + } |
| + |
| + if status, out := createInstance(t, fx, token, `{"host":"acme.go.tito.io","region":"ie"}`); status != http.StatusCreated { |
| + t.Fatalf("create = %d %s", status, out) |
| + } |
| + vars := envVars(t, fx, "acme") |
| + if _, ok := vars["JELLY_S3_BUCKET"]; !ok { |
| + t.Fatalf("acme bundle = %v, want the app's own JELLY_S3 prefix", varNames(vars)) |
| + } |
| + if _, ok := vars["CARLOS_STORE_BUCKET"]; ok { |
| + t.Fatalf("acme bundle also carries the default prefix: %v", varNames(vars)) |
| + } |
| +} |
| + |
| +func varNames(m map[string]appconfig.Var) []string { |
| + out := make([]string, 0, len(m)) |
| + for k := range m { |
| + out = append(out, k) |
| + } |
| + return out |
| +} |
| + |
| +// Declaring the policy is refused in the two states where it could only ever |
| +// be a trap — a deployment with no object storage, and an account nobody |
| +// approved. It fails once here, to the person, instead of once per signup in |
| +// a door whose caller is a machine. |
| +func TestEnableRefusesAPolicyItCannotHonour(t *testing.T) { |
| + t.Run("no backend", func(t *testing.T) { |
| + fx, _, token, _ := privateStorageFixture(t, "code-priv-enable-nobackend") |
| + fx.srv.cfg.AppsDomain = "oncarlos.test" |
| + fx.srv.cfg.StoreProvisioner = nil |
| + status, _, out := rawReq(t, http.MethodPut, enableURL(fx, "bab", "titogo"), token, `{"private_storage":true}`) |
| + if status != http.StatusServiceUnavailable { |
| + t.Fatalf("enable = %d %s, want 503", status, out) |
| + } |
| + if !strings.Contains(out, "CARLOS_STORE_BACKEND") { |
| + t.Errorf("refusal %q does not name the variable", out) |
| + } |
| + assertNoPolicy(t, fx) |
| + }) |
| + t.Run("unapproved account", func(t *testing.T) { |
| + fx, token := instanceFixture(t, "code-priv-enable-unapproved") |
| + fx.srv.cfg.AppsDomain = "oncarlos.test" |
| + fx.srv.cfg.StoreProvisioner = objstore.NewMemProvisioner("eu-west-1") |
| + status, _, out := rawReq(t, http.MethodPut, enableURL(fx, "bab", "titogo"), token, `{"private_storage":true}`) |
| + if status != http.StatusForbidden { |
| + t.Fatalf("enable = %d %s, want 403", status, out) |
| + } |
| + if !strings.Contains(out, "carlos store apply") { |
| + t.Errorf("refusal %q does not name the way out", out) |
| + } |
| + assertNoPolicy(t, fx) |
| + }) |
| +} |
| + |
| +func assertNoPolicy(t *testing.T, fx *consoleFixture) { |
| + t.Helper() |
| + m, err := provision.GetManifest(context.Background(), fx.rel, "bab", "titogo") |
| + if err == nil && m.PrivateStorage { |
| + t.Fatal("a refused enable wrote the policy anyway") |
| + } |
| +} |
| + |
| +// Turning it OFF is never refused, and the asymmetry is the point: an |
| +// account whose entitlement was later declined must be able to get its |
| +// create door working again without waiting on an operator. |
| +func TestEnableTurningPrivateStorageOffIsNeverRefused(t *testing.T) { |
| + fx, token := instanceFixture(t, "code-priv-enable-off") |
| + fx.srv.cfg.AppsDomain = "oncarlos.test" |
| + enableInstances(t, fx, "titogo", provision.Manifest{ |
| + Schema: 1, App: "titogo", Domain: "bab.oncarlos.test", PrivateStorage: true}) |
| + // No backend, no entitlement — the state the ON path refuses in. |
| + fx.srv.cfg.StoreProvisioner = nil |
| + |
| + status, _, out := rawReq(t, http.MethodPut, enableURL(fx, "bab", "titogo"), token, `{"private_storage":false}`) |
| + if status != http.StatusOK { |
| + t.Fatalf("enable off = %d %s, want 200", status, out) |
| + } |
| + m, err := provision.GetManifest(context.Background(), fx.rel, "bab", "titogo") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if m.PrivateStorage { |
| + t.Fatal("the policy survived being turned off") |
| + } |
| +} |
| + |
| +// An enable that never mentions storage preserves the policy — the same |
| +// absent-vs-explicit split channel and seed already have. A `carlos |
| +// instances enable -channel beta` that silently unset a tenant's isolation |
| +// is the failure this pins. |
| +func TestEnablePreservesPrivateStorage(t *testing.T) { |
| + fx, _, token, _ := privateStorageFixture(t, "code-priv-enable-preserve") |
| + fx.srv.cfg.AppsDomain = "oncarlos.test" |
| + enableInstances(t, fx, "titogo", provision.Manifest{ |
| + Schema: 1, App: "titogo", Domain: "bab.oncarlos.test", PrivateStorage: true}) |
| + |
| + status, _, out := rawReq(t, http.MethodPut, enableURL(fx, "bab", "titogo"), token, `{"channel":"beta"}`) |
| + if status != http.StatusOK { |
| + t.Fatalf("enable = %d %s, want 200", status, out) |
| + } |
| + var res struct { |
| + Channel string `json:"channel"` |
| + PrivateStorage bool `json:"private_storage"` |
| + } |
| + if err := json.Unmarshal([]byte(out), &res); err != nil { |
| + t.Fatal(err) |
| + } |
| + if res.Channel != "beta" || !res.PrivateStorage { |
| + t.Fatalf("reply = %+v, want channel beta with the policy intact", res) |
| + } |
| + m, err := provision.GetManifest(context.Background(), fx.rel, "bab", "titogo") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if !m.PrivateStorage { |
| + t.Fatal("a channel-only enable dropped the private-storage policy") |
| + } |
| +} |
| + |
| +// Declaring the policy stays a PERSON's act — the wall that makes this a |
| +// better answer than widening the instance role. An app token creates |
| +// instances all day and cannot write the manifest they are created under. |
| +func TestPrivateStoragePolicyStaysAHumanAct(t *testing.T) { |
| + fx, _, token, _ := privateStorageFixture(t, "code-priv-human") |
| + fx.srv.cfg.AppsDomain = "oncarlos.test" |
| + appToken := mintInstanceToken(t, fx, token, "titogo") |
| + |
| + status, _, out := rawReq(t, http.MethodPut, enableURL(fx, "bab", "titogo"), appToken, `{"private_storage":true}`) |
| + if status != http.StatusForbidden && status != http.StatusUnauthorized { |
| + t.Fatalf("app token enable = %d %s, want a refusal", status, out) |
| + } |
| + assertNoPolicy(t, fx) |
| +} |
| + |
| +// mintInstanceToken is `carlos instances token` — the service credential a |
| +// signup path holds. |
| +func mintInstanceToken(t *testing.T, fx *consoleFixture, personToken, app string) string { |
| + t.Helper() |
| + u := fx.ts.URL + "/api/cli/apps/bab/" + app + "/instance-token" |
| + status, _, out := rawReq(t, http.MethodPost, u, personToken, `{}`) |
| + if status != http.StatusCreated { |
| + t.Fatalf("mint instance token = %d %s", status, out) |
| + } |
| + var res struct { |
| + Token string `json:"token"` |
| + } |
| + if err := json.Unmarshal([]byte(out), &res); err != nil { |
| + t.Fatal(err) |
| + } |
| + return res.Token |
| +} |
diff --git a/internal/console/apistore.go b/internal/console/apistore.go| index 79a8fced7..07019da19 100644 |
| --- a/internal/console/apistore.go |
| +++ b/internal/console/apistore.go |
| @@ -398,6 +398,17 @@ func (s *Server) handleAPIStoreCreate(w http.ResponseWriter, r *http.Request) { |
| } |
| ctx, cancel := context.WithTimeout(r.Context(), storeTimeout) |
| defer cancel() |
| + // The environment-store ceiling (issue #412 §8) is enforced HERE as well |
| + // as on the automatic path. A cap that only the create-an-instance door |
| + // respected would be a cap anyone could step over by hand, and the |
| + // footprint it bounds — a bucket and an IAM user per store, against |
| + // IAM's per-account limits — does not care which door minted it. |
| + // Re-declaring a store that already exists is never capped; storeCeiling |
| + // carries that carve-out. |
| + if ref := s.storeCeiling(ctx, acct, app, env); ref != nil { |
| + jsonError(w, ref.Status, ref.Msg) |
| + return |
| + } |
| rec, etag, err := objstore.GetIn(ctx, s.cfg.Releases, acct.Sqid, app, env) |
| if err != nil { |
| // Logged, not returned: a store error carries bucket names, endpoints |
diff --git a/internal/console/instanceprivatestore.go b/internal/console/instanceprivatestore.go| new file mode 100644 |
| index 000000000..d68dd3cf1 |
| --- /dev/null |
| +++ b/internal/console/instanceprivatestore.go |
| @@ -0,0 +1,301 @@ |
| +// The app-declared private-storage policy (issue #412, spec |
| +// 2026-09-22-instance-private-storage-design.md): an app may say once, on |
| +// its instances manifest, that every instance it creates gets a config |
| +// environment of its own and an object store bound to that environment. |
| +// |
| +// The whole point is the SPLIT. Declaring the policy is |
| +// `handleAPIInstanceEnable` — person-only, zero Set, no service credential |
| +// reaches it. Spending it is `handleAPIInstanceCreate`, which an |
| +// `instance`-role credential can already reach. So a public signup path |
| +// provisions a tenant's own bucket without ever holding the power to create |
| +// one, which is what makes this a better answer than widening the role |
| +// (#410, which this makes unnecessary). |
| +// |
| +// Nothing here is a second implementation of provisioning. The walk this |
| +// file drives is the one `handleAPIStoreProvision` drives, with the same |
| +// lock, the same seal key, the same timeout and the same |
| +// GrantSourceAccountApproval — factored so the automatic path and the |
| +// member's own `carlos store create --environment` cannot drift. |
| +package console |
| + |
| +import ( |
| + "context" |
| + "errors" |
| + "fmt" |
| + "log" |
| + "net/http" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/appconfig" |
| + "github.com/carlosframework/platform/internal/objstore" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// storeMaxPerAppDefault is the ceiling on ENVIRONMENT stores under one app |
| +// when the deployment names none (spec §8, issue #412's point 3). |
| +// |
| +// A bucket plus an IAM user per instance is a bigger footprint than an |
| +// instance, and IAM's limits are per cloud account — so a signup path that |
| +// can mint them needs a bar it stops at. 100 is the number the issue itself |
| +// reaches for when it asks that `store status` and the console's |
| +// Environments view stay readable "at a hundred of them". |
| +// |
| +// Per APP rather than per account, and that is a cost decision stated as |
| +// such: the count is one prefix listing of the app, which the create door |
| +// can afford on every request. A deployment-wide count would mean listing |
| +// every app's stores on every create. |
| +const storeMaxPerAppDefault = 100 |
| + |
| +// storeMaxPerApp is the ceiling in force. Zero in the config means "not |
| +// configured", not "no stores allowed": a cap of zero would turn an |
| +// unset variable into an app that can have no environment store at all. |
| +func (s *Server) storeMaxPerApp() int { |
| + if s.cfg.StoreMaxPerApp > 0 { |
| + return s.cfg.StoreMaxPerApp |
| + } |
| + return storeMaxPerAppDefault |
| +} |
| + |
| +// storeCeiling refuses an app that has reached its environment-store cap. |
| +// |
| +// env is the environment ABOUT to be given a store, and a name that already |
| +// has one is never refused: the cap is on the FOOTPRINT, and re-declaring |
| +// an existing store adds no bucket and no principal. Without that carve-out |
| +// a retry of a create that failed just after provisioning would be turned |
| +// away by the very ceiling it had reached, with no way forward at all. |
| +// |
| +// A listing failure is a refusal, not a pass. This is a spend gate; reading |
| +// "how many are there already" as "none" on a bad round trip is the one |
| +// error direction that costs money. |
| +func (s *Server) storeCeiling(ctx context.Context, acct *Account, app, env string) *doorRefusal { |
| + if env == "" { |
| + return nil // the app's own store is not one of the capped collection |
| + } |
| + envs, err := objstore.StoreEnvironments(ctx, s.cfg.Releases, acct.Sqid, app) |
| + if err != nil { |
| + log.Printf("console: store ceiling %s/%s: %v", acct.Sqid, app, err) |
| + return &doorRefusal{Status: http.StatusBadGateway, |
| + Msg: "could not count this app's stores — try again"} |
| + } |
| + for _, e := range envs { |
| + if e == env { |
| + return nil |
| + } |
| + } |
| + max := s.storeMaxPerApp() |
| + if len(envs) < max { |
| + return nil |
| + } |
| + return &doorRefusal{Status: http.StatusConflict, |
| + Msg: fmt.Sprintf("%s already has %d environment stores, which is this deployment's limit of %d — destroy one in the console, or ask an operator to raise CARLOS_STORE_MAX_PER_APP", |
| + app, len(envs), max)} |
| +} |
| + |
| +// instanceStoreNames refuses an environment that cannot carry a store, for |
| +// an app under the policy (spec §6). |
| +// |
| +// derived says where the name came from — the host's first label, or the |
| +// caller's own `environment` — and it changes only the sentence, never the |
| +// rails. The first label is the derived name because it is the one |
| +// predictable from the host, already unique within the app (the record key |
| +// IS the host), and readable in `store status` at a hundred of them, which |
| +// is #412's point 1. |
| +// |
| +// Three refusals, all BEFORE anything is written, because each of them is |
| +// otherwise discovered somewhere worse: |
| +// |
| +// - a DNS label is not an environment name. Labels run to 63 characters |
| +// and environment names to 32, and "default" is a perfectly legal label |
| +// that names the app's own bundle rather than a named overlay. |
| +// - the bucket name is S3's 63. The walk checks this itself, but reaching |
| +// it there means an instance record already written over an estate that |
| +// cannot exist — and the walk's own refusal arrives as a 502 the caller |
| +// cannot act on, not a 400 naming the part to shorten. |
| +// - the principal name is IAM's 64, for the same reason. An environment |
| +// store's name carries three components, so a long enough (account, |
| +// app, environment) is simply unnameable. |
| +// |
| +// Each sentence names the part the caller can shorten, because from a |
| +// signup path the host label is usually the only one they control. |
| +func (s *Server) instanceStoreNames(acct *Account, app, env string, derived bool) *doorRefusal { |
| + source := "the environment you named" |
| + fix := "choose a shorter environment name" |
| + if derived { |
| + source = "the host's first label" |
| + fix = "choose a different host label" |
| + } |
| + if !appconfig.ValidEnvironmentName(env) { |
| + return &doorRefusal{Status: http.StatusBadRequest, |
| + Msg: fmt.Sprintf("%s gives every instance its own storage, which needs %s to be a usable environment name — %q is not (want ^[a-z0-9-]{1,32}$, and not \"default\"). Please %s", app, source, env, fix)} |
| + } |
| + bucket := objstore.MintedBucketIn(s.cfg.StoreDeployment, acct.Sqid, app, env) |
| + if !objstore.ValidBucketName(bucket) { |
| + return &doorRefusal{Status: http.StatusBadRequest, |
| + Msg: fmt.Sprintf("this instance's bucket would be named %q, which is %d characters and not a legal S3 name — %s", bucket, len(bucket), fix)} |
| + } |
| + user := objstore.UserNameIn(acct.Sqid, app, env) |
| + if !objstore.ValidUserName(user) { |
| + return &doorRefusal{Status: http.StatusBadRequest, |
| + Msg: fmt.Sprintf("this instance's IAM user would be named %q, which is %d characters, over AWS's %d — %s, or shorten the app name", user, len(user), objstore.MaxIAMUserName, fix)} |
| + } |
| + return nil |
| +} |
| + |
| +// instanceStoreProvision declares and provisions one environment's store |
| +// under the account's standing entitlement, and answers w itself on every |
| +// refusal. The caller has already proved membership and spent its write |
| +// budget; what is left is the store plane's own gates, in the order |
| +// handleAPIStoreProvision applies them. |
| +// |
| +// actor is the create door's principal — a person's address or the service |
| +// credential's `app:<sqid>/<app>` audit actor — and it lands in the grant |
| +// block, so a bucket minted by a signup path names the signup path. |
| +// |
| +// The env prefix is INHERITED from the app's own store when it has one. An |
| +// app that declared `carlos store create --env-prefix JELLY_S3` reads those |
| +// names in its code, and an instance store delivering CARLOS_STORE_BUCKET |
| +// into the same app would be a second vocabulary for one thing. With no app |
| +// store declared, the default prefix is taken — what a fresh declaration |
| +// would have taken anyway. |
| +// |
| +// ctx must be the REQUEST context, not a short-deadline derivative of it: |
| +// the walk below takes its own storeGrantTimeout, which is longer than the |
| +// create door's own deadline, and a walk that inherited a fifteen-second |
| +// budget would abandon a bucket it had already paid for. |
| +func (s *Server) instanceStoreProvision(ctx context.Context, w http.ResponseWriter, acct *Account, app, env, actor string) (*objstore.Record, bool) { |
| + if s.cfg.StoreProvisioner == nil { |
| + // The manifest carries a policy this deployment cannot honour. The |
| + // enable door refuses to write one in this state (§5), so reaching |
| + // here means the backend was unwired afterwards — a configuration |
| + // fact, named as one. |
| + jsonError(w, http.StatusServiceUnavailable, |
| + "object storage is not configured for this deployment (CARLOS_STORE_BACKEND), and "+app+" gives every instance its own storage — no instance can be created until one is wired or the policy is turned off (carlos instances enable --private-storage=false)") |
| + return nil, false |
| + } |
| + preCtx, preCancel := context.WithTimeout(ctx, storeTimeout) |
| + defer preCancel() |
| + if _, ok := s.storeProvisionApproval(preCtx, w, acct); !ok { |
| + return nil, false |
| + } |
| + pub, ok := s.storeSealKey(w) |
| + if !ok { |
| + return nil, false |
| + } |
| + // The same lock handleAPIStoreProvision takes, keyed the same way, so a |
| + // signup creating an instance and a member running `carlos store create |
| + // --environment` against the same name serialise against each other. |
| + defer s.storeWalks.lock(acct.Sqid, app, env)() |
| + // The walk's deadline starts AFTER the lock, for the reason that door |
| + // states: a request that queued behind a full-length walk would |
| + // otherwise reach the backend holding a spent context. |
| + walkCtx, cancel := context.WithTimeout(ctx, storeGrantTimeout) |
| + defer cancel() |
| + // Re-read under the lock: a queued request may have waited while an |
| + // operator declined the account. |
| + e, ok := s.storeProvisionApproval(walkCtx, w, acct) |
| + if !ok { |
| + return nil, false |
| + } |
| + // The declaration the walk requires. GrantIn refuses a store nobody |
| + // declared (ErrNotDeclared) because the env prefix is the member's |
| + // choice and a store with nowhere to be delivered is not one to invent |
| + // — under this policy the APP made that choice, once, on the manifest, |
| + // so the declaration is written here rather than demanded of a caller |
| + // who has no verb to write it with. |
| + prefix, ok := s.storeInheritedPrefix(walkCtx, w, acct, app) |
| + if !ok { |
| + return nil, false |
| + } |
| + rec, etag, err := objstore.GetIn(walkCtx, s.cfg.Releases, acct.Sqid, app, env) |
| + if err != nil { |
| + log.Printf("console: instance store read %s/%s/%s: %v", acct.Sqid, app, env, err) |
| + jsonError(w, http.StatusBadGateway, "could not read this instance's store record — try again") |
| + return nil, false |
| + } |
| + if rec == nil { |
| + rec = &objstore.Record{} |
| + } |
| + // Only when absent. A re-run over an existing declaration must not |
| + // re-stamp its author and timestamp, and must not move a prefix the app |
| + // is already reading — that is the one rewrite handleAPIStoreCreate |
| + // refuses with a 409, and this path has no member to show it to. |
| + if rec.Declared == nil { |
| + rec.Declared = &objstore.Declared{EnvPrefix: prefix, At: time.Now().UTC(), By: actor} |
| + if _, err := objstore.PutIn(walkCtx, s.cfg.Releases, acct.Sqid, app, env, rec, etag); err != nil { |
| + if errors.Is(err, release.ErrCASConflict) { |
| + jsonError(w, http.StatusConflict, |
| + "this instance's store record changed while it was being created — try again") |
| + return nil, false |
| + } |
| + log.Printf("console: instance store declare %s/%s/%s: %v", acct.Sqid, app, env, err) |
| + jsonError(w, http.StatusBadGateway, "could not declare this instance's store — nothing was created; try again") |
| + return nil, false |
| + } |
| + } |
| + out, err := objstore.GrantIn(walkCtx, s.cfg.Releases, s.cfg.StoreProvisioner, s.cfg.StoreDeployment, |
| + pub, acct.Sqid, app, env, e.Decision.By, objstore.GrantSourceAccountApproval, "", time.Now()) |
| + if err != nil { |
| + // Logged, not returned: a walk error carries bucket names, IAM ARNs |
| + // and request ids. The sentence the caller gets is the one they can |
| + // act on, and it is honest about what is now true — the declaration |
| + // may stand, the instance does not, and a retry of the SAME host |
| + // resumes this exact store because its name came from the host. |
| + log.Printf("console: instance store provision %s/%s/%s: %v", acct.Sqid, app, env, err) |
| + jsonError(w, http.StatusBadGateway, |
| + "could not provision this instance's storage, so the instance was not created — run the same command again to resume it") |
| + return nil, false |
| + } |
| + // Audited from here as well as from the store doors, because this is the |
| + // one path where a bucket is minted by a caller that is not a person: |
| + // an account reading its own rows has to be able to see that its signup |
| + // path spent money, and by whom it was authorised. |
| + s.audit(acct, app, actor, "store-provision", |
| + "provisioned "+storeBucketName(out)+" for instance environment "+env+" (instance create)") |
| + return out, true |
| +} |
| + |
| +// storeInheritedPrefix reads the app's own store for the variable-name |
| +// prefix a new environment store should deliver under. Absent is not an |
| +// error — an app with no store of its own is the ordinary shape for a |
| +// private-storage app, and takes the default. |
| +func (s *Server) storeInheritedPrefix(ctx context.Context, w http.ResponseWriter, acct *Account, app string) (string, bool) { |
| + rec, _, err := objstore.Get(ctx, s.cfg.Releases, acct.Sqid, app) |
| + if err != nil { |
| + log.Printf("console: instance store prefix %s/%s: %v", acct.Sqid, app, err) |
| + jsonError(w, http.StatusBadGateway, "could not read this app's store settings — try again") |
| + return "", false |
| + } |
| + return storeEffectivePrefix(rec), true |
| +} |
| + |
| +// instancePrivateStorageAllowed is the enable door's half: the two states in |
| +// which declaring the policy would be a trap rather than a policy (spec §5). |
| +// |
| +// A manifest that says "every instance gets a bucket" on a deployment with |
| +// no object storage, or in an account nobody approved, does not fail when it |
| +// is written. It fails once per signup, afterwards, in a door whose caller |
| +// is a machine — and the two sentences below are the only two that name the |
| +// fix. So they are said here, to the person, at the moment they declare it. |
| +// |
| +// Turning the policy OFF is never refused, and that asymmetry is the point: |
| +// an account whose entitlement was later declined must be able to get its |
| +// create door working again without an operator. |
| +func (s *Server) instancePrivateStorageAllowed(ctx context.Context, w http.ResponseWriter, acct *Account) bool { |
| + if s.cfg.StoreProvisioner == nil { |
| + jsonError(w, http.StatusServiceUnavailable, |
| + "object storage is not configured for this deployment (CARLOS_STORE_BACKEND) — an app cannot be given per-instance storage here") |
| + return false |
| + } |
| + e, _, err := objstore.GetEntitlement(ctx, s.cfg.Releases, acct.Sqid) |
| + if err != nil { |
| + log.Printf("console: instances enable entitlement %s: %v", acct.Sqid, err) |
| + jsonError(w, http.StatusBadGateway, "could not read this account's storage approval — try again") |
| + return false |
| + } |
| + if !e.Approved() { |
| + jsonError(w, http.StatusForbidden, |
| + "this account is not approved for object storage, so its instances cannot be given storage of their own — run `carlos store apply` and ask an operator to approve it first") |
| + return false |
| + } |
| + return true |
| +} |
diff --git a/internal/console/server.go b/internal/console/server.go| index 7c1aedd22..ed4352157 100644 |
| --- a/internal/console/server.go |
| +++ b/internal/console/server.go |
| @@ -592,6 +592,17 @@ type Config struct { |
| // lives — a walk that guessed would create the next bucket somewhere |
| // else, and the record would say so afterwards. |
| StoreDeployment objstore.Deployment |
| + // StoreMaxPerApp caps how many ENVIRONMENT stores one app may have |
| + // (CARLOS_STORE_MAX_PER_APP; 0 = storeMaxPerAppDefault). The app's own |
| + // store is not one of them and is never capped. |
| + // |
| + // It exists because #412 made a store something a signup path can mint: |
| + // a bucket plus an IAM user per instance is a bigger footprint than an |
| + // instance, and IAM's limits are per cloud account, so the automatic |
| + // path needs a bar it stops at. Enforced on the MANUAL door too |
| + // (handleAPIStoreCreate) — a ceiling only the automatic path respected |
| + // would be a ceiling anyone could step over by hand. |
| + StoreMaxPerApp int |
| // StoreOperators are the addresses allowed to drive the GRANT door, and |
| // only that one (CARLOS_STORE_OPERATORS). Its own list rather than |
| // console ownership on purpose: granting CREATES backend estate and |
diff --git a/internal/provision/provision.go b/internal/provision/provision.go| index 7c68ed826..e958c2dc8 100644 |
| --- a/internal/provision/provision.go |
| +++ b/internal/provision/provision.go |
| @@ -190,6 +190,26 @@ type Manifest struct { |
| Channel string `json:"channel,omitempty"` |
| Regions []string `json:"regions,omitempty"` |
| Seed []string `json:"seed,omitempty"` |
| + |
| + // PrivateStorage is the app's standing policy that every instance |
| + // created under this manifest gets a config environment of its own and |
| + // an object store bound to that environment (issue #412, spec |
| + // 2026-09-22-instance-private-storage-design.md §5). Declared once, by |
| + // a signed-in person, at `carlos instances enable --private-storage`; |
| + // spent by the create door, which an `instance`-role service credential |
| + // can already reach. That split is the point: the policy widens no |
| + // role, so a signup path provisions a tenant's bucket without ever |
| + // holding the power to create one. |
| + // |
| + // A plain bool, not a pointer, and the difference from Record's |
| + // Hibernate and StatusNamed is worth stating because their reason does |
| + // not apply here. Those are CONVERGED columns — a box re-reads them |
| + // every pass, so "absent" has to be distinguishable from an explicit |
| + // false or the first pass after deploy acts on a value nobody wrote. |
| + // This is read at one door, at create time, and false IS the behaviour |
| + // every manifest in a production bucket already has. There is nothing |
| + // for absence to mean that false does not already mean. |
| + PrivateStorage bool `json:"private_storage,omitempty"` |
| } |
| // ManifestKey, RecordsPrefix and RecordKey are the bucket key conventions |
diff --git a/internal/provision/provision_test.go b/internal/provision/provision_test.go| index 1ab77449c..e233f5bce 100644 |
| --- a/internal/provision/provision_test.go |
| +++ b/internal/provision/provision_test.go |
| @@ -1115,3 +1115,50 @@ func TestLayoutJournalPath(t *testing.T) { |
| t.Errorf("LayoutJournalPath = %q", got) |
| } |
| } |
| + |
| +// PrivateStorage is additive and its ZERO VALUE is the legacy behaviour: a |
| +// manifest written before the field existed — every manifest in a |
| +// production bucket — must read as an app whose instances share the app's |
| +// storage, which is what those apps do (issue #412, spec §5). |
| +func TestManifestPrivateStorageIsAdditive(t *testing.T) { |
| + ctx := context.Background() |
| + s := release.NewDirStore(t.TempDir()) |
| + |
| + // The bytes an older console wrote: no such key at all. |
| + legacy := []byte(`{"schema":1,"app":"tito","domain":"go.tito.io","channel":"stable"}`) |
| + if _, err := s.Put(ctx, ManifestKey("bad", "tito"), legacy, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := GetManifest(ctx, s, "bad", "tito") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if got.PrivateStorage { |
| + t.Fatal("a manifest written before the field reads as private-storage — every app in production would start minting buckets") |
| + } |
| + |
| + got.PrivateStorage = true |
| + if err := PutManifest(ctx, s, "bad", "tito", *got); err != nil { |
| + t.Fatal(err) |
| + } |
| + back, err := GetManifest(ctx, s, "bad", "tito") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if !back.PrivateStorage { |
| + t.Fatalf("policy did not survive a round trip: %+v", back) |
| + } |
| + // omitempty: an app without the policy writes no key, so nothing an old |
| + // reader parses changes shape. |
| + back.PrivateStorage = false |
| + if err := PutManifest(ctx, s, "bad", "tito", *back); err != nil { |
| + t.Fatal(err) |
| + } |
| + raw, _, err := s.Get(ctx, ManifestKey("bad", "tito")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if strings.Contains(string(raw), "private_storage") { |
| + t.Errorf("an app without the policy still writes the key: %s", raw) |
| + } |
| +} |