amadan

carlos / platform Public

console: the hourly scan pass, started from serve()

Its own ticker, not storagePass's. Draft 2 wanted to ride that one; it
returns as soon as the day's usage object exists, so it runs once per
UTC day — a sweep advancing one page a day would never finish a bucket
of any size. Nor is it folded into economicsPass, whose single
goroutine already runs census, statements and billing and already
worries about a slow step delaying the others.

The interval is a floor for COMPLETED sweeps only. A sweep still in
flight is always due, which is the only way a bucket larger than one
budget ever finishes; a finished one is left alone so LastComplete
stays stable long enough to quote in an audit answer rather than being
rebuilt from scratch every hour.

One app's failure does not stop the pass. Giving up on the first
unreachable bucket would leave every store after it in listing order
permanently unswept — an ordering-dependent coverage gap, the worst
kind to diagnose.

Boot refuses a scanner with no store to scan, and refuses an unknown
engine name the way storeBackend refuses an unknown backend: a typo
that silently disabled scanning would leave an operator believing a
compliance control is running when nothing is. Durations and byte
counts are fatal on garbage for the same reason — a deployment that
meant hourly and typed it wrong should be told, not quietly given 24h.

Started from serve(), never from NewServer, so constructing a Server
stays goroutine-free.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJkJkxZZLgW8X8A5tk69Sq
Paul Campbell pushed by paul@keymail.dev 10dc492b85b52ddd4134969f565debc318dadd6e parent de4e2f0
4 files changed, +540 −0
  • cmd/console/main.go +83 −0
  • internal/console/scanpass.go +206 −0
  • internal/console/scanpass_test.go +227 −0
  • internal/console/server.go +24 −0
diff --git a/cmd/console/main.go b/cmd/console/main.go
index bda5c2259..b7f3b6e47 100644
--- a/cmd/console/main.go
+++ b/cmd/console/main.go
@@ -78,6 +78,7 @@ import (
"net/http"
"os"
"sort"
+ "strconv"
"strings"
"time"
@@ -258,6 +259,7 @@ func serve() {
}
st := openConsoleDB()
storeProvisioner, storeDeployment := storeBackend()
+ storeScanner := storeScannerFor(storeDeployment)
mailDep := mailDeployment()
mailProvisioner, dnsWriter := mailBackends(mailDep, mail.NewAWSProvisioner, dnsrec.NewRoute53Writer)
srv, err := console.NewServer(console.Config{
@@ -280,6 +282,10 @@ func serve() {
RedirectHosts: redirectHosts(),
DenyDomains: denyDomains(),
StoreProvisioner: storeProvisioner,
+ StoreScanner: storeScanner,
+ ScanBudget: envDuration("CARLOS_STORE_SCAN_BUDGET", time.Minute),
+ ScanInterval: envDuration("CARLOS_STORE_SCAN_INTERVAL", 24*time.Hour),
+ ScanMaxSigAge: envDuration("CARLOS_CLAMD_MAX_SIG_AGE", 168*time.Hour),
StoreDeployment: storeDeployment,
StoreOperators: storeOperators(),
// CARLOS_ACME_DELEGATION_ZONE — the same variable the edge reads
@@ -328,6 +334,11 @@ func serve() {
// /fleet/architecture (internal/console/infrapass.go). Same placement
// rule as the economics pass: started here, never in NewServer.
go srv.StartInfraPass(context.Background())
+ // The object-storage scan pass advances each opted-in store's coverage
+ // (internal/console/scanpass.go). Same placement rule again: started
+ // here, never in NewServer. It returns immediately when no engine is
+ // configured, which is every deployment that has not opted in.
+ go srv.StartScanPass(context.Background())
addr := envOr("CONSOLE_ADDR", "127.0.0.1:8090")
ln, err := net.Listen("tcp", addr)
if err != nil {
@@ -729,6 +740,78 @@ func denyDomains() []string {
return out
}
+// storeScannerFor builds the object-storage virus-scanning engine from the
+// environment, or returns nil — which is the ordinary shape. Scanning is
+// opt-in and off by default: a deployment that sets nothing gets a nil
+// Scanner, no pass goroutine, and doors that refuse.
+//
+// CARLOS_STORE_SCANNER "clamav" (the only value this build accepts)
+// CARLOS_CLAMD_SOCKET default /run/clamav/clamd.ctl
+// CARLOS_CLAMD_MAX_BYTES default 25 MB — mirror clamd's own StreamMaxLength
+//
+// The unknown-value posture is storeBackend's, and for the same reason: a
+// typo that silently disabled scanning would leave an operator believing a
+// compliance control is on when nothing is running.
+func storeScannerFor(d objstore.Deployment) objstore.Scanner {
+ engine := strings.TrimSpace(os.Getenv("CARLOS_STORE_SCANNER"))
+ if engine == "" {
+ return nil
+ }
+ if engine != "clamav" {
+ // GuardDuty is deliberately not wired in this build (spec §4.2): its
+ // 25-protected-buckets-per-account-per-Region quota is not
+ // adjustable, so it cannot be the primary engine for a platform whose
+ // unit of tenancy is a bucket. The spec §7.4 refusal for
+ // "guardduty with a non-empty endpoint" is unreachable while this is
+ // the only accepted value, which is why there is no dead branch for
+ // it here.
+ log.Fatalf("console: CARLOS_STORE_SCANNER %q is not an engine this build knows — the only value is \"clamav\" (leave it unset for a deployment with no object scanning)", engine)
+ }
+ // A scanner with no store to scan is a misconfiguration, and one loud
+ // failure at boot beats a pass that silently finds nothing forever.
+ if d.Backend == "" {
+ log.Fatal("console: CARLOS_STORE_SCANNER is set but CARLOS_STORE_BACKEND is not — there are no object stores on this deployment to scan")
+ }
+ socket := envOr("CARLOS_CLAMD_SOCKET", "/run/clamav/clamd.ctl")
+ maxBytes := envBytes("CARLOS_CLAMD_MAX_BYTES", 25<<20)
+ sc, err := objstore.NewClamAVScanner(context.Background(), d.Region, d.Endpoint, socket, maxBytes)
+ if err != nil {
+ log.Fatal(err)
+ }
+ return sc
+}
+
+// envDuration reads a Go duration ("90s", "12h"). A value that does not parse
+// is fatal rather than silently defaulted: a deployment that meant to scan
+// hourly and typed it wrong should be told, not quietly given 24h.
+func envDuration(k string, def time.Duration) time.Duration {
+ v := strings.TrimSpace(os.Getenv(k))
+ if v == "" {
+ return def
+ }
+ d, err := time.ParseDuration(v)
+ if err != nil {
+ log.Fatalf("console: %s %q is not a duration (e.g. 90s, 30m, 24h): %v", k, v, err)
+ }
+ if d <= 0 {
+ log.Fatalf("console: %s %q must be positive", k, v)
+ }
+ return d
+}
+
+// envBytes reads a plain byte count. Same fatal-on-garbage posture.
+func envBytes(k string, def int64) int64 {
+ v := strings.TrimSpace(os.Getenv(k))
+ if v == "" {
+ return def
+ }
+ n, err := strconv.ParseInt(v, 10, 64)
+ if err != nil || n <= 0 {
+ log.Fatalf("console: %s %q must be a positive number of bytes", k, v)
+ }
+ return n
+}
+
func envOr(k, def string) string {
if v := os.Getenv(k); v != "" {
return v
diff --git a/internal/console/scanpass.go b/internal/console/scanpass.go
new file mode 100644
index 000000000..b2eb689e8
--- /dev/null
+++ b/internal/console/scanpass.go
@@ -0,0 +1,206 @@
+// The scan pass (spec §7.1): the goroutine that drives objstore.RunSweep over
+// every store that has opted in.
+//
+// Its own ticker, deliberately. Draft 2 wanted to ride storagePass; that was
+// wrong — storagePass returns once the day's usage object exists
+// (economicspass.go:137-140), so it runs once per UTC day, and a sweep that
+// advances one page a day would never finish a bucket of any size. It is also
+// not folded into economicsPass, whose single goroutine already runs census,
+// statements and billing and already worries about a slow step delaying the
+// others.
+package console
+
+import (
+ "context"
+ "errors"
+ "log"
+ "strings"
+ "time"
+
+ "github.com/carlosframework/platform/internal/objstore"
+)
+
+// scanPassInterval is how often the pass ticks. The re-sweep FLOOR is a
+// separate number (Config.ScanInterval): ticking hourly is what lets a
+// part-finished sweep make progress every hour, while a finished one is left
+// alone until it is genuinely stale.
+const scanPassInterval = time.Hour
+
+const (
+ defaultScanBudget = time.Minute
+ defaultScanInterval = 24 * time.Hour
+)
+
+// StartScanPass runs scanPass immediately and then hourly until ctx is done —
+// StartInfraPass's shape (infrapass.go:49).
+func (s *Server) StartScanPass(ctx context.Context) {
+ if s.cfg.Releases == nil || s.cfg.StoreScanner == nil {
+ return // no bucket, or nobody opted in: nothing to tick for
+ }
+ t := time.NewTicker(scanPassInterval)
+ defer t.Stop()
+ for {
+ if err := s.scanPass(ctx); err != nil {
+ log.Printf("console: scan pass: %v", err)
+ }
+ select {
+ case <-ctx.Done():
+ return
+ case <-t.C:
+ }
+ }
+}
+
+func (s *Server) scanBudget() time.Duration {
+ if s.cfg.ScanBudget > 0 {
+ return s.cfg.ScanBudget
+ }
+ return defaultScanBudget
+}
+
+func (s *Server) scanInterval() time.Duration {
+ if s.cfg.ScanInterval > 0 {
+ return s.cfg.ScanInterval
+ }
+ return defaultScanInterval
+}
+
+// scanPass gives every enabled store one budget of scanning.
+//
+// Enumeration is storagePass's (economicspass.go:150-181): ListDirs("apps/"),
+// per-account ListDirs, per-app objstore.Get. Hourly, that is 24x that pass's
+// listing cost — accepted for v1 and named here so it is a known cost rather
+// than a surprise. An index is the fix if it bites.
+//
+// One app's failure does not stop the others. A store whose engine is
+// unreachable, whose record names a different engine, or whose bucket has gone
+// is logged and skipped: the pass exists to make progress on what it can.
+func (s *Server) scanPass(ctx context.Context) error {
+ if s.cfg.Releases == nil || s.cfg.StoreScanner == nil {
+ return nil
+ }
+ acctPrefixes, err := s.cfg.Releases.ListDirs(ctx, "apps/")
+ if err != nil {
+ return err
+ }
+ budget := s.scanBudget()
+ interval := s.scanInterval()
+
+ for _, acctPrefix := range acctPrefixes {
+ acct := strings.TrimSuffix(strings.TrimPrefix(acctPrefix, "apps/"), "/")
+ appPrefixes, err := s.cfg.Releases.ListDirs(ctx, acctPrefix)
+ if err != nil {
+ return err
+ }
+ for _, appPrefix := range appPrefixes {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ app := strings.TrimSuffix(strings.TrimPrefix(appPrefix, acctPrefix), "/")
+ if err := s.scanOneApp(ctx, acct, app, budget, interval); err != nil {
+ log.Printf("console: scan pass: %s/%s: %v", acct, app, err)
+ }
+ }
+ }
+ return nil
+}
+
+// scanOneApp decides whether this app is due and, if so, sweeps it.
+func (s *Server) scanOneApp(ctx context.Context, acct, app string, budget, interval time.Duration) error {
+ rec, _, err := objstore.Get(ctx, s.cfg.Releases, acct, app)
+ if err != nil {
+ return err
+ }
+ // Absent is (nil, "", nil) — objstore.Get's own idiom — so this is "no
+ // store", not a failure.
+ if rec == nil || rec.Provisioned == nil || rec.Scanning == nil || rec.Scanning.Enabled == nil {
+ return nil
+ }
+
+ due, err := s.storeIsDue(ctx, acct, app, interval)
+ if err != nil {
+ return err
+ }
+ if !due {
+ return nil
+ }
+
+ lister, err := s.storeLister(ctx, rec.Provisioned)
+ if err != nil {
+ return err
+ }
+ _, err = objstore.RunSweep(ctx, s.cfg.Releases, lister, s.cfg.StoreScanner, acct, app, budget)
+ return err
+}
+
+// storeIsDue answers whether this store wants work now.
+//
+// A sweep already in flight is always due — that is how a bucket bigger than
+// one budget ever finishes. Only a COMPLETED sweep waits out the interval, so
+// LastComplete stays stable long enough to be quoted in an audit answer
+// instead of being rebuilt from scratch every hour.
+func (s *Server) storeIsDue(ctx context.Context, acct, app string, interval time.Duration) (bool, error) {
+ side, _, err := objstore.GetSidecar(ctx, s.cfg.Releases, acct, app)
+ if err != nil {
+ // A sidecar we cannot read is not a reason to re-sweep blindly: it is
+ // a schema or ownership refusal that wants an operator, and sweeping
+ // anyway would overwrite whatever it was trying to protect.
+ return false, err
+ }
+ if side == nil {
+ return true, nil // never swept
+ }
+ if side.Current != nil {
+ return true, nil // mid-sweep: keep going
+ }
+ if side.NotApplicable {
+ // Client-encrypted. Re-checking hourly costs a listing and can never
+ // change until the member re-attests, which rewrites the record.
+ return false, nil
+ }
+ if side.LastComplete == nil {
+ return true, nil
+ }
+ return time.Since(side.LastComplete.At) >= interval, nil
+}
+
+// storeLister routes through the test seam or builds a real S3 lister, the
+// way sizeStore already does for the storage sweep.
+func (s *Server) storeLister(ctx context.Context, p *objstore.Provisioned) (objstore.ObjectLister, error) {
+ if s.storeListerFor != nil {
+ return s.storeListerFor(ctx, p)
+ }
+ client, err := objstore.S3ClientFor(ctx, p.Region, p.Endpoint)
+ if err != nil {
+ return nil, err
+ }
+ return objstore.NewObjectLister(client), nil
+}
+
+// errNoScanner is what the doors return when scanning was never configured.
+var errNoScanner = errors.New("console: no scanning engine is configured on this deployment")
+
+// errNoStore is a sweep asked for on an app that has no provisioned store.
+var errNoStore = errors.New("console: this app has no object store")
+
+// sweepNow drives one sweep on demand, for the operator verb. It shares
+// everything with the ticker except the schedule — spec §7.1's guarantee is
+// "at most one driver RECORDS", not that two never run: the sidecar's CAS
+// guard makes the loser stand down.
+func (s *Server) sweepNow(ctx context.Context, acct, app string) (*objstore.ScanSidecar, error) {
+ if s.cfg.StoreScanner == nil {
+ return nil, errNoScanner
+ }
+ rec, _, err := objstore.Get(ctx, s.cfg.Releases, acct, app)
+ if err != nil {
+ return nil, err
+ }
+ if rec == nil || rec.Provisioned == nil {
+ return nil, errNoStore
+ }
+ lister, err := s.storeLister(ctx, rec.Provisioned)
+ if err != nil {
+ return nil, err
+ }
+ return objstore.RunSweep(ctx, s.cfg.Releases, lister, s.cfg.StoreScanner, acct, app, s.scanBudget())
+}
diff --git a/internal/console/scanpass_test.go b/internal/console/scanpass_test.go
new file mode 100644
index 000000000..77339ad87
--- /dev/null
+++ b/internal/console/scanpass_test.go
@@ -0,0 +1,227 @@
+package console
+
+import (
+ "context"
+ "errors"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/objstore"
+)
+
+// countingLister serves one small page from an in-memory key set and counts
+// how many times it was asked. The count is the assertion for "did a sweep
+// actually run", which a timestamp cannot answer reliably at test speed.
+type countingLister struct {
+ keys []string
+ calls int
+ err error
+}
+
+func (l *countingLister) ListPage(_ context.Context, _, startAfter string, _ int32) ([]objstore.ScanObject, bool, error) {
+ l.calls++
+ if l.err != nil {
+ return nil, false, l.err
+ }
+ var objs []objstore.ScanObject
+ for _, k := range l.keys {
+ if k > startAfter {
+ objs = append(objs, objstore.ScanObject{Key: k, ETag: "etag-" + k, Size: 10})
+ }
+ }
+ return objs, false, nil
+}
+
+// refusingLister fails the test if it is ever consulted.
+type refusingLister struct{ t *testing.T }
+
+func (l *refusingLister) ListPage(context.Context, string, string, int32) ([]objstore.ScanObject, bool, error) {
+ l.t.Fatal("a bucket was listed that should never have been swept")
+ return nil, false, nil
+}
+
+// enableStore writes a provisioned, scanning-enabled record for acct/app.
+func enableStore(t *testing.T, fx *consoleFixture, acct, app string, enabled bool) {
+ t.Helper()
+ ctx := context.Background()
+ rec := &objstore.Record{
+ Provisioned: &objstore.Provisioned{
+ Bucket: "carlos-" + acct + "-" + app,
+ Region: "eu-west-1",
+ },
+ }
+ if enabled {
+ rec.Scanning = &objstore.ScanBlock{
+ Requested: &objstore.ScanRequested{At: time.Now().UTC(), By: "m@example.com"},
+ Enabled: &objstore.ScanEnabled{Engine: "clamav", At: time.Now().UTC(), By: "op@example.com"},
+ }
+ }
+ if _, err := objstore.Put(ctx, fx.rel, acct, app, rec, ""); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func withLister(fx *consoleFixture, l objstore.ObjectLister) {
+ fx.srv.storeListerFor = func(context.Context, *objstore.Provisioned) (objstore.ObjectLister, error) {
+ return l, nil
+ }
+}
+
+// No scanner configured: the pass does nothing and touches no backend. This is
+// every deployment that has not opted in, which is all of them by default.
+func TestScanPassInertWithoutScanner(t *testing.T) {
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ enableStore(t, fx, fixtureAcct, fixtureApp, true)
+ withLister(fx, &refusingLister{t: t})
+
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatal(err)
+ }
+ side, _, err := objstore.GetSidecar(ctx, fx.rel, fixtureAcct, fixtureApp)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if side != nil {
+ t.Fatalf("a deployment with no engine wrote coverage: %+v", side)
+ }
+}
+
+// The pass sweeps only stores that are provisioned AND enabled. A store nobody
+// opted in for must not acquire a scan record as a side effect of living in
+// the same bucket as one that did.
+func TestScanPassSweepsOnlyEnabledStores(t *testing.T) {
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ fx.srv.cfg.StoreScanner = objstore.NewMemScanner()
+
+ enableStore(t, fx, fixtureAcct, fixtureApp, true)
+ enableStore(t, fx, fixtureAcct, "unopted", false)
+ withLister(fx, &countingLister{keys: []string{"a.txt", "b.txt"}})
+
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatal(err)
+ }
+
+ side, _, err := objstore.GetSidecar(ctx, fx.rel, fixtureAcct, fixtureApp)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if side == nil || side.LastComplete == nil || side.LastComplete.Objects != 2 {
+ t.Fatalf("the enabled store was not swept: %+v", side)
+ }
+
+ other, _, err := objstore.GetSidecar(ctx, fx.rel, fixtureAcct, "unopted")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if other != nil {
+ t.Fatalf("a store nobody opted in for was swept: %+v", other)
+ }
+}
+
+// A completed sweep is not restarted before the interval, so LastComplete
+// stays stable long enough to quote in an audit answer instead of being
+// rebuilt from scratch every hour.
+func TestScanPassHonoursReSweepInterval(t *testing.T) {
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ fx.srv.cfg.StoreScanner = objstore.NewMemScanner()
+ fx.srv.cfg.ScanInterval = time.Hour
+ enableStore(t, fx, fixtureAcct, fixtureApp, true)
+
+ l := &countingLister{keys: []string{"a.txt"}}
+ withLister(fx, l)
+
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatal(err)
+ }
+ if l.calls != 1 {
+ t.Fatalf("first pass listed %d times, want 1", l.calls)
+ }
+
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatal(err)
+ }
+ if l.calls != 1 {
+ t.Fatalf("a completed sweep was restarted inside the interval: %d listings", l.calls)
+ }
+
+ // Once the interval has passed, it sweeps again — otherwise coverage
+ // would be a one-time claim that ages silently.
+ fx.srv.cfg.ScanInterval = time.Nanosecond
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatal(err)
+ }
+ if l.calls != 2 {
+ t.Fatalf("a stale sweep was not restarted: %d listings", l.calls)
+ }
+}
+
+// One app's failure does not stop the rest. A pass that gave up on the first
+// unreachable bucket would leave every store after it in the listing order
+// permanently unswept — an ordering-dependent coverage gap, which is the worst
+// kind to diagnose.
+func TestScanPassContinuesPastOneAppsFailure(t *testing.T) {
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ fx.srv.cfg.StoreScanner = objstore.NewMemScanner()
+
+ enableStore(t, fx, fixtureAcct, "aaa-broken", true)
+ enableStore(t, fx, fixtureAcct, fixtureApp, true)
+
+ good := &countingLister{keys: []string{"a.txt"}}
+ fx.srv.storeListerFor = func(_ context.Context, p *objstore.Provisioned) (objstore.ObjectLister, error) {
+ if p.Bucket == "carlos-"+fixtureAcct+"-aaa-broken" {
+ return nil, errors.New("endpoint down")
+ }
+ return good, nil
+ }
+
+ if err := fx.srv.scanPass(ctx); err != nil {
+ t.Fatalf("one app's failure failed the whole pass: %v", err)
+ }
+ side, _, err := objstore.GetSidecar(ctx, fx.rel, fixtureAcct, fixtureApp)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if side == nil || side.LastComplete == nil {
+ t.Fatal("a healthy store after a broken one was never swept")
+ }
+}
+
+// A sweep still in flight ignores the interval entirely. This is the only way
+// a bucket larger than one budget ever finishes: if a part-done sweep waited
+// out the re-sweep floor, a big store would advance one page a day.
+func TestScanPassAlwaysContinuesAnInFlightSweep(t *testing.T) {
+ ctx := context.Background()
+ fx := newConsoleFixture(t)
+ fx.srv.cfg.ScanInterval = 365 * 24 * time.Hour
+
+ side := &objstore.ScanSidecar{
+ Current: &objstore.Sweep{StartedAt: time.Now().UTC(), At: time.Now().UTC(), After: "a.txt"},
+ }
+ if _, err := objstore.PutSidecar(ctx, fx.rel, fixtureAcct, fixtureApp, side, ""); err != nil {
+ t.Fatal(err)
+ }
+
+ due, err := fx.srv.storeIsDue(ctx, fixtureAcct, fixtureApp, fx.srv.scanInterval())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if !due {
+ t.Fatal("an in-flight sweep was parked until the re-sweep interval")
+ }
+}
+
+// The on-demand verb refuses plainly when no engine is configured, rather than
+// reporting a sweep that never happened.
+func TestSweepNowRefusesWithoutScanner(t *testing.T) {
+ fx := newConsoleFixture(t)
+ enableStore(t, fx, fixtureAcct, fixtureApp, true)
+
+ _, err := fx.srv.sweepNow(context.Background(), fixtureAcct, fixtureApp)
+ if !errors.Is(err, errNoScanner) {
+ t.Fatalf("err = %v, want errNoScanner", err)
+ }
+}
diff --git a/internal/console/server.go b/internal/console/server.go
index 4b866b258..d0c6cf6d8 100644
--- a/internal/console/server.go
+++ b/internal/console/server.go
@@ -530,6 +530,24 @@ type Config struct {
// engine released before its bucket goes, so a nil scanner over an enabled
// record is a refusal rather than a skip.
StoreScanner objstore.Scanner
+ // ScanBudget bounds how much scanning ONE app gets per tick
+ // (CARLOS_STORE_SCAN_BUDGET, default 60s). The arithmetic matters and is
+ // stated in the spec: an hourly tick at 60s is 24 minutes of scanning per
+ // app per day, so at a few hundred objects a minute over INSTREAM a
+ // million-object bucket takes months. The default is honest for small and
+ // medium stores; a deployment with large ones raises it or shortens the
+ // tick.
+ ScanBudget time.Duration
+ // ScanInterval is the floor before a COMPLETED sweep is started again
+ // (CARLOS_STORE_SCAN_INTERVAL, default 24h). A sweep still in flight
+ // ignores it — that is how a bucket bigger than one budget finishes.
+ ScanInterval time.Duration
+ // ScanMaxSigAge is how old an engine's signature database may be before
+ // `store scan status` warns (CARLOS_CLAMD_MAX_SIG_AGE, default 168h). A
+ // clean verdict from three-month-old signatures is not the same claim as
+ // a fresh one, and an audit answer that cannot tell them apart is worth
+ // less than it looks.
+ ScanMaxSigAge time.Duration
// StoreDeployment is the deployment-wide half of every store: which
// backend, which region and endpoint buckets are made in, and the prefix
// minted bucket names carry. Threaded into every walk so this config is
@@ -648,6 +666,12 @@ type Server struct {
// (sizeProvisionedStore, economicspass.go); tests inject a fake so a
// sweep can be proven to degrade ("unswept") without live AWS.
storeSizer func(ctx context.Context, p *objstore.Provisioned) (int64, error)
+
+ // storeListerFor is the scan pass's test seam for listing a provisioned
+ // bucket, the same shape and for the same reason as storeSizer above: the
+ // sweep is driven against a MemProvisioner's buckets with no S3 and no
+ // credentials.
+ storeListerFor func(ctx context.Context, p *objstore.Provisioned) (objstore.ObjectLister, error)
// ceCosts is billPass's test/fake seam for Cost Explorer (cepull.go,
// economicspass.go): nil means "use the real, lazily-built CE client"
// (s.realCECosts); tests inject a fake so a pull can be proven to