Snapshots Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A document that has been edited past maxRetainedUpdates keeps accepting edits, and replaying one costs a snapshot plus a short tail instead of its whole history.

Architecture: The Rust core gains a state export — one Yjs update encoding the entire document — and a verifier that checks a candidate against the document it came from without leaving the core. Store.ApplyUpdate takes that snapshot from the replayed state before applying the incoming update, verifies it, and writes it to a new doc_snapshots table every 256 updates, deleting the one it supersedes. Store.replay and GET /d/{sqid}/updates both read snapshot-then-tail; a snapshot is never authoritative, so anything unreadable falls back to a full replay of the log.

Tech Stack: Rust yrs 0.23.5 compiled to wasm32 and run under wazero, Go 1.26, SQLite through GORM, JS yjs 13.6.27 unchanged.

Spec: docs/superpowers/specs/2026-09-01-snapshots-design.md. Read it before Task 1 — every decision below argues from one of its D-numbers, and the two most dangerous mistakes in this work (where the snapshot is taken from, and which yrs call encodes it) are both spec decisions rather than implementation choices.

Global Constraints

• CGO_ENABLED=0. One static binary. No new Go module requirements.
• No entropy from the CRDT library. Doc::new, Options::with_client_id and friends are banned by clippy disallowed-methods. A new Doc takes both a guid and a client id from its caller — including the throwaway document Task 2's verifier builds.
• The core module must have zero host imports (core/tests/no_host_imports.rs).
• internal/core's ABI check is exact-match: the wasm artifact and the host move together, and a stale artifact fails at load. Rebuild it after any Rust change.
• Never call sessions.UserID; never use scope.Owned. The AST guard in internal/lint fails the build.
• Store.ApplyUpdate's owner re-derivation and its MAX(seq)+INSERT transaction are not to be weakened, reordered, or moved.
• The fixtures in fixtures/ are the contract, not an output.
• Run .amadan/ci.d/10-gate before reporting any task complete. Exit 0 is green.
• Environment: GOCACHE must point somewhere writable ($TMPDIR/go-build); the default is read-only under the agent sandbox and fails as "setup failed", which reads like a build break and is not one. The Rust toolchain is at ~/.cargo/bin, not on a default PATH. A fresh worktree needs npm ci in fixtures/.

File Structure

| File | Responsibility | |---|---| | core/src/snapshot.rs (new) | Encode a document's whole state as one update; verify a candidate against a live document. The only place that knows either operation. | | core/src/lib.rs | Re-export snapshot_bytes / verify_snapshot. | | core/src/abi.rs | docs_core_snapshot_len / _read, docs_core_verify_snapshot; ABI 3 → 4. | | core/tests/snapshot.rs (new) | Round-trip, merge-equivalence, and verifier tests in Rust. | | internal/core/core.go | abiVersion 3 → 4; (*Handle).Snapshot, (*Handle).VerifySnapshot. | | internal/docs/migrations/0004_doc_snapshots.sql (new) | The table. | | internal/docs/models.go | DocSnapshot. | | internal/docs/snapshots.go (new) | Read the newest usable snapshot; write one and supersede its predecessor. The only file that knows the cadence. | | internal/docs/store.go | replay reads snapshot-then-tail; ApplyUpdate takes and writes the snapshot. | | internal/docs/handlers.go | updatesGet serves snapshot-then-tail, and ?raw=1. | | fixtures/live-updates.mjs | One retry through ?raw=1 when applying fetched bytes throws. | | fixtures/editor-entry.js | Pass the raw flag through; same fallback on the initial load. |

---

Task 1: The core can encode a document's whole state

Without this there is nothing to store. The danger here is picking the wrong yrs call: two neighbours look right and are not (spec §3).

Files:

• Create: core/src/snapshot.rs, core/tests/snapshot.rs
• Modify: core/src/lib.rs, core/src/abi.rs

Interfaces:

• Produces: pub fn snapshot_bytes(doc: &Doc) -> Vec<u8> — the whole state as one update.
• Produces ABI: docs_core_snapshot_len(handle) -> i32, docs_core_snapshot_read(handle, out_ptr, out_len) -> i32, same convention and error codes as the existing text/html/json pairs.

• [ ] Step 1: Write the failing test

core/tests/snapshot.rs:

use yrs::{Doc, Options, ReadTxn, Transact, XmlFragmentPrelim, GetString};
use docs_core::{snapshot_bytes, fragment_text};

fn doc_with(client_id: u64) -> Doc {
    Doc::with_options(Options::with_guid_and_client_id("test-guid".into(), client_id))
}

#[test]
fn a_snapshot_replayed_into_an_empty_doc_reproduces_the_document() {
    let source = doc_with(1);
    let update = std::fs::read("../fixtures/vectors/fragment/simple.bin").unwrap();
    source.transact_mut().apply_update(yrs::Update::decode_v1(&update).unwrap()).unwrap();

    let snap = snapshot_bytes(&source);

    let rebuilt = doc_with(2);
    rebuilt.transact_mut().apply_update(yrs::Update::decode_v1(&snap).unwrap()).unwrap();

    assert_eq!(fragment_text(&rebuilt), fragment_text(&source));
    assert!(!fragment_text(&rebuilt).is_empty(), "fixture produced no text");
}

• [ ] Step 2: Run it and watch it fail

Run: PATH="$HOME/.cargo/bin:$PATH" cargo test --manifest-path core/Cargo.toml --test snapshot Expected: FAIL — snapshot_bytes is not defined.

• [ ] Step 3: Implement snapshot_bytes

core/src/snapshot.rs:

//! Encoding a whole document as one update, and checking that the
//! result is the document it claims to be.
//!
//! The call below is `encode_state_as_update_v1` against an EMPTY state
//! vector, and which call this is matters more than anything else in
//! this file. Against the empty vector it encodes every block, the
//! entire delete set, and — through yrs's own `merge_pending_v1` — the
//! pending queue, meaning updates whose dependencies never arrived and
//! which `apply_update` accepted and parked. That makes the result
//! merge-equivalent to the log it replaces, not merely
//! render-equivalent: every clock range and every tombstone survives,
//! so a later concurrent merge integrates against a snapshot exactly as
//! it would against the full history.
//!
//! Two neighbours are traps, and both look like this one:
//!
//! - `encode_diff_v1(&StateVector::default())` silently drops the
//!   pending queue (its own doc comment says so). A document holding an
//!   update whose dependency arrives later heals under a full replay and
//!   does NOT heal under a diff-built snapshot — and no render
//!   comparison can see the difference, because pending content is
//!   invisible in both documents.
//! - yrs's own `Snapshot` type and `encode_state_from_snapshot` are a
//!   different feature (historical views, requiring `skip_gc`) that
//!   happens to share this module's name.
use yrs::{Doc, ReadTxn, StateVector, Transact};

/// The whole document, as one update.
pub fn snapshot_bytes(doc: &Doc) -> Vec<u8> {
    doc.transact().encode_state_as_update_v1(&StateVector::default())
}

Add to core/src/lib.rs:

mod snapshot;
pub use snapshot::snapshot_bytes;

• [ ] Step 4: Run the test and watch it pass

Run: PATH="$HOME/.cargo/bin:$PATH" cargo test --manifest-path core/Cargo.toml --test snapshot Expected: PASS.

• [ ] Step 5: Add the ABI pair

In core/src/abi.rs, beside json_string:

/// The whole document as one update. Bytes, not a string — the host
/// reads it through the same len/read convention as the three
/// renderers, which is why this is shaped like them.
pub fn snapshot_vec(handle: u32) -> Option<Vec<u8>> {
    get_doc(handle).map(|doc| snapshot_bytes(&doc))
}

and the two exports, copying the docs_core_json_len / docs_core_json_read bodies exactly, substituting snapshot_vec for json_string and s.as_ptr() reading from the Vec<u8>:

#[no_mangle]
pub extern "C" fn docs_core_snapshot_len(handle: u32) -> i32 {
    match snapshot_vec(handle) {
        Some(v) => v.len().try_into().unwrap_or(i32::MAX),
        None => ERR_NO_DOC,
    }
}

/// # Safety
///
/// `out_ptr` and `out_len` must describe a byte range that lies entirely
/// within this module's linear memory and is valid for writes for the
/// duration of this call.
#[no_mangle]
pub unsafe extern "C" fn docs_core_snapshot_read(handle: u32, out_ptr: u32, out_len: u32) -> i32 {
    let v = match snapshot_vec(handle) {
        Some(v) => v,
        None => return ERR_NO_DOC,
    };
    if v.len() > out_len as usize {
        return ERR_BUFFER_TOO_SMALL;
    }
    if !in_bounds(out_ptr, out_len) {
        return ERR_INVALID_LENGTH;
    }
    std::ptr::copy_nonoverlapping(v.as_ptr(), out_ptr as *mut u8, v.len());
    v.len().try_into().unwrap_or(i32::MAX)
}

• [ ] Step 6: Bump the ABI version to 4 in core/src/abi.rs, and leave the existing exports untouched.

• [ ] Step 7: Build the artifact and run the Rust suite

Run: PATH="$HOME/.cargo/bin:$PATH" ./scripts/build-wasm.sh && PATH="$HOME/.cargo/bin:$PATH" DOCS_REQUIRE_WASM=1 cargo test --manifest-path core/Cargo.toml Expected: PASS, including no_host_imports.

• [ ] Step 8: Commit — git add core/ && git commit

---

Task 2: The core can tell a good snapshot from a bad one

Spec D5. This runs inside the core so the comparison is against Rust structs rather than encoded bytes, whose encoding is not canonical.

Files:

• Modify: core/src/snapshot.rs, core/src/abi.rs, core/src/lib.rs
• Test: core/tests/snapshot.rs

Interfaces:

• Produces: pub fn verify_snapshot(doc: &Doc, candidate: &[u8], guid: &str, client_id: u64) -> bool
• Produces ABI: docs_core_verify_snapshot(handle, ptr, len) -> i32 — 0 match, 1 mismatch, ERR_DECODE undecodable, ERR_NO_DOC unknown handle.

• [ ] Step 1: Write the failing tests

Append to core/tests/snapshot.rs:

#[test]
fn a_snapshot_of_this_document_verifies() {
    let source = doc_with(1);
    let update = std::fs::read("../fixtures/vectors/fragment/simple.bin").unwrap();
    source.transact_mut().apply_update(yrs::Update::decode_v1(&update).unwrap()).unwrap();
    let snap = snapshot_bytes(&source);
    assert!(docs_core::verify_snapshot(&source, &snap, "verify-guid", 99));
}

#[test]
fn a_snapshot_of_a_different_document_does_not_verify() {
    let source = doc_with(1);
    source.transact_mut()
        .apply_update(yrs::Update::decode_v1(
            &std::fs::read("../fixtures/vectors/fragment/simple.bin").unwrap()).unwrap()).unwrap();

    let other = doc_with(1);
    other.transact_mut()
        .apply_update(yrs::Update::decode_v1(
            &std::fs::read("../fixtures/vectors/fragment/marks.bin").unwrap()).unwrap()).unwrap();
    let wrong = snapshot_bytes(&other);

    assert!(!docs_core::verify_snapshot(&source, &wrong, "verify-guid", 99));
}

#[test]
fn garbage_does_not_verify() {
    let source = doc_with(1);
    assert!(!docs_core::verify_snapshot(&source, &[0xff, 0x00, 0x13, 0x37], "verify-guid", 99));
}

• [ ] Step 2: Run and watch all three fail (verify_snapshot undefined).

• [ ] Step 3: Implement it

Append to core/src/snapshot.rs:

/// True when `candidate`, replayed into an empty document, is the same
/// document as `doc`.
///
/// Three comparisons, none of them on encoded bytes: update encoding is
/// not canonical (merge boundaries vary), so byte equality would report
/// false mismatches and no cheap byte-level proof exists. State vector
/// equality says both documents saw the same clock ranges from the same
/// clients; delete set equality says they agree on every deletion;
/// `fragment_json` equality says they render to the same structure,
/// which catches formatting-only divergence that a text comparison
/// misses at identical cost.
///
/// This is deliberately not a proof. What survives all three is "yrs
/// corrupted content while preserving every clock range, every deletion
/// and the full structural render", and that is a narrow enough residual
/// to accept.
///
/// What it CANNOT catch, stated because the temptation is to believe it
/// can: a snapshot taken from a stale state verifies perfectly, because
/// it is compared against the very document that produced it. That
/// failure is closed by taking the snapshot from the replayed state
/// (the spec's D3), not here.
///
/// guid/client_id are the caller's, never generated here — the clippy
/// disallowed-methods rule against drawing entropy from the CRDT library
/// applies to this throwaway document exactly as it does to a real one.
pub fn verify_snapshot(doc: &Doc, candidate: &[u8], guid: &str, client_id: u64) -> bool {
    let update = match yrs::Update::decode_v1(candidate) {
        Ok(u) => u,
        Err(_) => return false,
    };
    let rebuilt = Doc::with_options(Options::with_guid_and_client_id(guid.into(), client_id));
    {
        let mut txn = rebuilt.transact_mut();
        if txn.apply_update(update).is_err() {
            return false;
        }
    }
    let (a, b) = (doc.transact(), rebuilt.transact());
    a.state_vector() == b.state_vector()
        && a.delete_set() == b.delete_set()
        && crate::fragment_json(doc) == crate::fragment_json(&rebuilt)
}

Export it from lib.rs (pub use snapshot::{snapshot_bytes, verify_snapshot};).

• [ ] Step 4: Run the tests and watch them pass.

• [ ] Step 5: Add the ABI export

/// 0 = the candidate is this document, 1 = it is not, ERR_DECODE = the
/// bytes are not an update at all, ERR_NO_DOC = no such handle.
///
/// # Safety
///
/// `ptr`/`len` must describe a readable byte range inside this module's
/// linear memory.
#[no_mangle]
pub unsafe extern "C" fn docs_core_verify_snapshot(handle: u32, ptr: u32, len: u32) -> i32 {
    if !in_bounds(ptr, len) {
        return ERR_INVALID_LENGTH;
    }
    let candidate = std::slice::from_raw_parts(ptr as *const u8, len as usize);
    match get_doc(handle) {
        None => ERR_NO_DOC,
        Some(doc) => {
            // The verifier's throwaway document takes a fixed guid and
            // client id rather than generating any: entropy inside the
            // core is banned (clippy disallowed-methods), and this
            // document is discarded before the call returns, so a fixed
            // pair is not merely acceptable but correct.
            if verify_snapshot(&doc, candidate, "docs-core-verify", 1) { 0 } else { 1 }
        }
    }
}

• [ ] Step 6: Rebuild the artifact, run the Rust suite, commit.

---

Task 3: The Go host can take and check a snapshot

Files:

• Modify: internal/core/core.go
• Test: internal/core/core_test.go

Interfaces:

• Produces: func (h *Handle) Snapshot(ctx context.Context) ([]byte, error)
• Produces: func (h *Handle) VerifySnapshot(ctx context.Context, candidate []byte) (bool, error)

• [ ] Step 1: Write the failing test

func TestSnapshotRoundTripsThroughAFreshHandle(t *testing.T) {
	rt := newTestRuntime(t)
	ctx := context.Background()

	h, err := rt.Open(ctx, "guid-a", 1)
	if err != nil {
		t.Fatalf("Open: %v", err)
	}
	defer h.Close(ctx)
	if err := h.Apply(ctx, fragmentFixture(t, "simple")); err != nil {
		t.Fatalf("Apply: %v", err)
	}
	want, err := h.Text(ctx)
	if err != nil {
		t.Fatalf("Text: %v", err)
	}

	snap, err := h.Snapshot(ctx)
	if err != nil {
		t.Fatalf("Snapshot: %v", err)
	}
	if len(snap) == 0 {
		t.Fatal("Snapshot returned no bytes")
	}

	fresh, err := rt.Open(ctx, "guid-b", 2)
	if err != nil {
		t.Fatalf("Open fresh: %v", err)
	}
	defer fresh.Close(ctx)
	if err := fresh.Apply(ctx, snap); err != nil {
		t.Fatalf("Apply snapshot: %v", err)
	}
	got, err := fresh.Text(ctx)
	if err != nil {
		t.Fatalf("Text: %v", err)
	}
	if got != want {
		t.Errorf("snapshot rebuilt %q, want %q", got, want)
	}
}

func TestVerifySnapshotAcceptsItsOwnAndRejectsAnother(t *testing.T) {
	rt := newTestRuntime(t)
	ctx := context.Background()
	h, _ := rt.Open(ctx, "guid-a", 1)
	defer h.Close(ctx)
	if err := h.Apply(ctx, fragmentFixture(t, "simple")); err != nil {
		t.Fatalf("Apply: %v", err)
	}
	snap, err := h.Snapshot(ctx)
	if err != nil {
		t.Fatalf("Snapshot: %v", err)
	}

	ok, err := h.VerifySnapshot(ctx, snap)
	if err != nil {
		t.Fatalf("VerifySnapshot: %v", err)
	}
	if !ok {
		t.Error("a document refused its own snapshot")
	}

	if ok, err := h.VerifySnapshot(ctx, []byte{0xff, 0x00}); err != nil || ok {
		t.Errorf("garbage verified: ok=%v err=%v", ok, err)
	}
}

Match the existing helpers in core_test.go for building a runtime and reading a fragment fixture; if they are named differently, use the existing names rather than adding new ones.

• [ ] Step 2: Run and watch it fail.

• [ ] Step 3: Bump abiVersion to 4 and implement

// Snapshot returns the document's entire state as one Yjs update — what
// doc_snapshots stores so a later replay can start from it instead of
// from the whole log (the snapshots design, D1).
//
// Bytes rather than a string, but it rides the same len/read machinery
// as Text/HTML/JSON, including their trap-and-retry: a Go string is a
// byte sequence and nothing here interprets it, so "snapshot" is simply
// a fourth renderer name to readOn.
func (h *Handle) Snapshot(ctx context.Context) ([]byte, error) {
	s, err := h.read(ctx, "snapshot")
	if err != nil {
		return nil, err
	}
	return []byte(s), nil
}

// VerifySnapshot asks the core whether candidate, replayed into an empty
// document, is this document — state vector, delete set and structural
// render (the snapshots design, D5). It is a smoke test rather than a
// proof, and it cannot detect a snapshot taken from a stale state; see
// core/src/snapshot.rs for both limits.
func (h *Handle) VerifySnapshot(ctx context.Context, candidate []byte) (bool, error) {
	// ... allocate candidate into guest memory exactly as Apply does,
	// call docs_core_verify_snapshot, free, and map: 0 -> true, 1 ->
	// false, ERR_DECODE -> false with no error (bad bytes are an answer,
	// not a failure), anything else -> error.
}

Write VerifySnapshot's body by copying Apply's allocate/call/free structure — it is the only existing export that takes a byte buffer, and its guest-memory handling, trap taxonomy and retry are what this needs too.

• [ ] Step 4: Rebuild the artifact, run go test ./internal/core/, commit.

---

Task 4: The table, and reading the newest usable snapshot

Files:

• Create: internal/docs/migrations/0004_doc_snapshots.sql, internal/docs/snapshots.go
• Modify: internal/docs/models.go
• Test: internal/docs/snapshots_test.go (new)

Interfaces:

• Produces: type DocSnapshot struct { ID, DocID, Seq int64; State []byte; ABIVersion uint32; CreatedAt time.Time }
• Produces: func (s *Store) newestSnapshot(ctx context.Context, docID int64) (*DocSnapshot, error) — nil, nil when there is none usable.
• Produces: const snapshotCadence = 256

• [ ] Step 1: Write the migration

0004_doc_snapshots.sql:

-- doc_snapshots holds one Yjs update per row encoding a document's
-- entire state as of `seq` (the snapshots design, D2). It is an
-- accelerator, never the source of truth: doc_updates remains complete
-- and authoritative, and a row here that cannot be read is discarded in
-- favour of replaying the log.
--
-- (doc_id, seq) is deliberately NOT unique. Two concurrent POSTs at the
-- cadence boundary can both snapshot the same replayed prefix, and those
-- rows are equivalent by construction, so a collision is not an error to
-- raise. "Newest usable" resolves as highest seq, then highest id.
--
-- abi_version records what produced the row, which is what makes
-- discarding a snapshot from an older core implementable rather than
-- merely promised.
--
-- Additive: creates a table, touches no existing row.
CREATE TABLE IF NOT EXISTS doc_snapshots (
  id          INTEGER PRIMARY KEY AUTOINCREMENT,
  doc_id      INTEGER NOT NULL,
  seq         INTEGER NOT NULL,
  state       BLOB    NOT NULL,
  abi_version INTEGER NOT NULL,
  created_at  DATETIME NOT NULL
);

CREATE INDEX IF NOT EXISTS idx_doc_snapshots_doc_seq
  ON doc_snapshots (doc_id, seq DESC, id DESC);

• [ ] Step 2: Write the failing test

internal/docs/snapshots_test.go:

func TestNewestSnapshotPrefersHighestSeqThenHighestID(t *testing.T) {
	app, err := NewForTest(t)
	if err != nil {
		t.Fatalf("NewForTest: %v", err)
	}
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Snapshots")

	write := func(seq int64, state string, abi uint32) {
		row := &DocSnapshot{DocID: doc.ID, Seq: seq, State: []byte(state), ABIVersion: abi, CreatedAt: time.Now().UTC()}
		if err := app.Store().db.WithContext(ctx).Create(row).Error; err != nil {
			t.Fatalf("insert snapshot: %v", err)
		}
	}
	write(10, "older", coreABIVersion)
	write(20, "first-at-20", coreABIVersion)
	write(20, "second-at-20", coreABIVersion)

	got, err := app.Store().newestSnapshot(ctx, doc.ID)
	if err != nil {
		t.Fatalf("newestSnapshot: %v", err)
	}
	if got == nil || string(got.State) != "second-at-20" {
		t.Fatalf("newestSnapshot returned %+v, want the second row at seq 20", got)
	}
}

func TestNewestSnapshotIgnoresAnotherABIsRows(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Old ABI")

	row := &DocSnapshot{DocID: doc.ID, Seq: 5, State: []byte("from an older core"), ABIVersion: coreABIVersion - 1, CreatedAt: time.Now().UTC()}
	if err := app.Store().db.WithContext(ctx).Create(row).Error; err != nil {
		t.Fatalf("insert: %v", err)
	}

	got, err := app.Store().newestSnapshot(ctx, doc.ID)
	if err != nil {
		t.Fatalf("newestSnapshot: %v", err)
	}
	if got != nil {
		t.Error("a snapshot from another ABI was treated as usable")
	}
}

func TestNewestSnapshotIsNilWhenThereAreNone(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "None")

	got, err := app.Store().newestSnapshot(ctx, doc.ID)
	if err != nil || got != nil {
		t.Errorf("newestSnapshot = (%v, %v), want (nil, nil)", got, err)
	}
}

• [ ] Step 3: Run and watch them fail.

• [ ] Step 4: Add the model and the reader

models.go:

// DocSnapshot is one document's entire state encoded as a single Yjs
// update, as of Seq. It is an accelerator for replay and for the update
// endpoint, never the source of truth — doc_updates keeps every row, and
// a snapshot that cannot be read is discarded in favour of replaying
// them (the snapshots design, D2).
//
// State is plaintext in this table, exactly as DocUpdate.UpdateBin is.
// Worth one caveat the sealing work should carry: a doc_updates row is a
// fragment of an edit, while this is the entire document in one blob.
//
// ABIVersion records the core that produced State. Without it, "discard
// a snapshot from an older ABI" is unenforceable prose.
type DocSnapshot struct {
	ID    int64 `gorm:"primaryKey"`
	DocID int64 `gorm:"column:doc_id;not null;index"`
	Seq   int64 `gorm:"column:seq;not null"`

	State      []byte `gorm:"column:state;not null"`
	ABIVersion uint32 `gorm:"column:abi_version;not null"`

	CreatedAt time.Time `gorm:"column:created_at;not null"`
}

snapshots.go:

package docs

// snapshotCadence is how many updates may accumulate past a snapshot
// before the next one is taken. 256 keeps a replay to a snapshot plus a
// short tail, which is what holds the retained-update count in a
// core.Handle clear of maxRetainedUpdates (4096) — about 16x of headroom.
//
// That headroom is conditional, and the condition is worth knowing:
// it holds while snapshot writes keep succeeding. Roughly fifteen
// consecutive failures — each logged at Error, so noisy rather than
// silent — walk the tail back to the cap.
const snapshotCadence = 256

// newestSnapshot returns the most recent usable snapshot for docID, or
// nil when there is none.
//
// "Usable" excludes rows written by a different core ABI: the state
// encoding is that core's business, and a row from another one is
// discarded rather than trusted. Ordering is seq DESC, id DESC — two
// concurrent POSTs at the cadence boundary may legitimately write
// equivalent snapshots at the same seq, so the tie resolves to the last
// row written rather than being treated as an error.
func (s *Store) newestSnapshot(ctx context.Context, docID int64) (*DocSnapshot, error) {
	var row DocSnapshot
	err := s.db.WithContext(ctx).
		Where("doc_id = ? AND abi_version = ?", docID, coreABIVersion).
		Order("seq DESC, id DESC").
		First(&row).Error
	if errors.Is(err, gorm.ErrRecordNotFound) {
		return nil, nil
	}
	if err != nil {
		return nil, fmt.Errorf("docs: newest snapshot for document %d: %w", docID, err)
	}
	return &row, nil
}

Export the ABI version from internal/core as core.ABIVersion and alias it here as coreABIVersion, rather than duplicating the number.

• [ ] Step 5: Register the migration in models.go's Schema — it is migrate.MustFromFS(migrationFS, "docs") over migrations/*.sql, so the new file is picked up by its number with no code change. Confirm by running the tests.

• [ ] Step 6: Run tests, then commit.

---

Task 5: Replay reads snapshot-then-tail, and writes one at the cadence

The heart of it, and the task carrying both defects the spec review found.

Files:

• Modify: internal/docs/store.go, internal/docs/snapshots.go
• Test: internal/docs/snapshots_test.go

Interfaces:

• Produces: func (s *Store) writeSnapshot(ctx context.Context, doc *Document, h *core.Handle, seq int64) error
• Changes: replay starts from the newest usable snapshot; ApplyUpdate takes the snapshot from the replayed state before applying the incoming update.

• [ ] Step 1: Write the failing tests — equivalence, the race, and the capped document

// The central property: a document replayed through snapshot-then-tail
// must render identically to one replayed from the whole log.
func TestSnapshotThenTailRendersIdenticallyToAFullReplay(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Equivalence")

	update := validUpdate(t)
	for i := 0; i < snapshotCadence+5; i++ {
		if _, err := app.Store().ApplyUpdate(ctx, doc, update); err != nil {
			t.Fatalf("ApplyUpdate %d: %v", i, err)
		}
	}
	withSnapshot, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML: %v", err)
	}

	// Delete every snapshot and read again: same document, full replay.
	if err := app.Store().db.WithContext(ctx).
		Where("doc_id = ?", doc.ID).Delete(&DocSnapshot{}).Error; err != nil {
		t.Fatalf("delete snapshots: %v", err)
	}
	fullReplay, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML after deleting snapshots: %v", err)
	}
	if withSnapshot != fullReplay {
		t.Error("snapshot-then-tail and full replay disagree about the document")
	}
}

// The spec's D3, first defect: a snapshot must never claim a seq whose
// content it does not contain. Asserted against the rendered document,
// never against the snapshot row — the row is exactly what a stale
// snapshot makes look right.
func TestConcurrentPostsAtTheCadenceBoundaryDoNotStrandContent(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Boundary")

	for i := 0; i < snapshotCadence-1; i++ {
		if _, err := app.Store().ApplyUpdate(ctx, doc, validUpdate(t)); err != nil {
			t.Fatalf("warmup %d: %v", i, err)
		}
	}

	var wg sync.WaitGroup
	for i := 0; i < 8; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			_, _ = app.Store().ApplyUpdate(ctx, doc, validUpdate(t))
		}()
	}
	wg.Wait()

	withSnapshot, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML: %v", err)
	}
	if err := app.Store().db.WithContext(ctx).
		Where("doc_id = ?", doc.ID).Delete(&DocSnapshot{}).Error; err != nil {
		t.Fatalf("delete snapshots: %v", err)
	}
	fullReplay, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML: %v", err)
	}
	if withSnapshot != fullReplay {
		t.Error("a snapshot written during concurrent writes stranded content the log still holds")
	}
}

// The spec's D3, second defect, and the case section 1 opens with: a
// document that is ALREADY at the cap when this ships. Driven through
// the new path a document never reaches the cap, so this builds the
// production shape directly — a full log and no snapshot — and asserts
// it recovers.
func TestADocumentAlreadyAtTheCapRecovers(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "At the cap")

	// Write rows straight into the log, bypassing ApplyUpdate, so no
	// snapshot is ever taken — exactly the state a document edited
	// before this feature shipped is in.
	update := validUpdate(t)
	rows := make([]DocUpdate, 0, core.MaxRetainedUpdates)
	for i := 1; i <= core.MaxRetainedUpdates; i++ {
		rows = append(rows, DocUpdate{DocID: doc.ID, Seq: int64(i), UpdateBin: update, CreatedAt: time.Now().UTC()})
	}
	if err := app.Store().db.WithContext(ctx).CreateInBatches(rows, 500).Error; err != nil {
		t.Fatalf("seed a full log: %v", err)
	}

	// The first write still fails — the handle is full before the
	// snapshot can help — but it must leave a snapshot behind.
	if _, err := app.Store().ApplyUpdate(ctx, doc, update); err == nil {
		t.Log("first write succeeded; acceptable if the cap was not reached")
	}
	snap, err := app.Store().newestSnapshot(ctx, doc.ID)
	if err != nil {
		t.Fatalf("newestSnapshot: %v", err)
	}
	if snap == nil {
		t.Fatal("a capped document got no snapshot, so it can never recover")
	}

	// The retry replays snapshot + empty tail and succeeds.
	if _, err := app.Store().ApplyUpdate(ctx, doc, update); err != nil {
		t.Errorf("retry after a snapshot existed still failed: %v", err)
	}
}

// A corrupted snapshot row must not corrupt the document: the log is
// authoritative, so an unreadable snapshot is discarded and the whole
// log replayed instead. Correct, and merely slower.
func TestACorruptSnapshotFallsBackToTheLog(t *testing.T) {
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Corrupt")
	for i := 0; i < snapshotCadence+2; i++ {
		if _, err := app.Store().ApplyUpdate(ctx, doc, validUpdate(t)); err != nil {
			t.Fatalf("ApplyUpdate %d: %v", i, err)
		}
	}
	want, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML: %v", err)
	}

	if err := app.Store().db.WithContext(ctx).Model(&DocSnapshot{}).
		Where("doc_id = ?", doc.ID).
		Update("state", []byte{0xff, 0x00, 0x13, 0x37}).Error; err != nil {
		t.Fatalf("corrupt the snapshot: %v", err)
	}

	got, err := app.Store().HTML(ctx, doc)
	if err != nil {
		t.Fatalf("HTML with a corrupt snapshot: %v", err)
	}
	if got != want {
		t.Error("a corrupt snapshot changed the document instead of being discarded")
	}
}

// The cliff itself, driven the long way round: more updates than
// maxRetainedUpdates through the real path, which must keep accepting
// them. Slow — thousands of wasm applies — so it is skipped under
// -short, but it is the one test that proves the headline claim rather
// than a proxy for it.
func TestADocumentPastTheRetainedCapKeepsAcceptingWrites(t *testing.T) {
	if testing.Short() {
		t.Skip("thousands of updates; run without -short")
	}
	app, _ := NewForTest(t)
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, _ := app.Store().Create(ctx, owner, "Past the cap")

	for i := 0; i < core.MaxRetainedUpdates+50; i++ {
		if _, err := app.Store().ApplyUpdate(ctx, doc, validUpdate(t)); err != nil {
			t.Fatalf("ApplyUpdate %d of %d: %v", i, core.MaxRetainedUpdates+50, err)
		}
	}
}

core.MaxRetainedUpdates and core.ABIVersion must be exported from internal/core for these tests and for snapshots.go; export them rather than duplicating 4096 and the version number.

replay's signature changes to return what the caller needs rather than making it ask again: func (s *Store) replay(ctx context.Context, doc *Document) (h *core.Handle, replayedSeq int64, lastSnapshotSeq int64, err error). Both seqs come from rows it has already read — the same no-second-query discipline the live-sync design's D2 sets, for the same reason.

• [ ] Step 2: Run all three and watch them fail.

• [ ] Step 3: Make replay read snapshot-then-tail

In store.go's replay, before loading updates: read newestSnapshot; if there is one, h.Apply(ctx, snap.State) and load only seq > snap.Seq. If applying it fails, log at Error, discard it, and fall back to loading the whole log — a snapshot is never authoritative.

• [ ] Step 4: Take and write the snapshot in ApplyUpdate, before Apply

After replay returns the handle and before h.Apply(ctx, update):

	// The snapshot is taken HERE — from the replayed state, before the
	// incoming update is applied — and the placement is the whole of its
	// correctness (the snapshots design, D3).
	//
	// Taking it after a successful Apply is wrong twice. The replay above
	// runs outside the transaction below, so a concurrent POST can commit
	// a row this handle never saw; a snapshot taken from that handle
	// would claim a seq whose content it does not hold, and D5's
	// verification cannot notice, because it compares the candidate
	// against this same stale handle. And a document already at the cap
	// never reaches an after-success hook at all — Apply refuses first —
	// so the one case this feature exists for would stay broken forever.
	//
	// The replayed prefix is complete by construction: seq is gapless
	// (MAX+1 under a single-writer pool) and replay read every row that
	// existed when it ran. So replayedSeq is unconditionally truthful.
	if replayedSeq-lastSnapshotSeq >= snapshotCadence {
		if err := s.writeSnapshot(ctx, doc, h, replayedSeq); err != nil {
			// Never fatal: the update below still stands, the log is
			// untouched, and the next POST tries again.
			s.logger.Error("docs: write snapshot", "doc_id", doc.ID, "seq", replayedSeq, "err", err)
		}
	}

replay returns replayedSeq and lastSnapshotSeq alongside the handle — read from the snapshot row it already loaded, never a second query.

• [ ] Step 5: Implement writeSnapshot

Take h.Snapshot(ctx), h.VerifySnapshot(ctx, candidate), refuse to write on a false verdict (log at Error and return an error), otherwise insert the row with core.ABIVersion and delete every older snapshot for that document — superseded snapshots are derived data, reproducible from the log, so removing them deletes no history and keeps this feature from costing one document-sized blob per 256 updates.

• [ ] Step 6: Run the three tests, plus the whole internal/docs suite, and watch them pass.

• [ ] Step 7: Commit.

---

Task 6: The update endpoint serves snapshot-then-tail, and ?raw=1

Files:

• Modify: internal/docs/handlers.go
• Test: internal/docs/cursor_test.go

• [ ] Step 1: Write the failing tests

// pastCadence builds a document with more updates than the cadence, so a
// snapshot exists, and returns it.
func pastCadence(t *testing.T, app *App, title string) *Document {
	t.Helper()
	ctx := context.Background()
	owner := app.TestPerson(t, "owner@example.com")
	doc, err := app.Store().Create(ctx, owner, title)
	if err != nil {
		t.Fatalf("Create: %v", err)
	}
	for i := 0; i < snapshotCadence+3; i++ {
		if _, err := app.Store().ApplyUpdate(ctx, doc, validUpdate(t)); err != nil {
			t.Fatalf("ApplyUpdate %d: %v", i, err)
		}
	}
	return doc
}

// A client far behind the snapshot receives the snapshot plus the tail
// rather than the whole log, and the header still names the highest seq
// the bytes actually reach.
func TestUpdatesGetServesSnapshotThenTail(t *testing.T) {
	app, err := NewForTest(t)
	if err != nil {
		t.Fatalf("NewForTest: %v", err)
	}
	doc := pastCadence(t, app, "Snapshot then tail")

	snap := serve(app, app.TestRequest(t, "owner@example.com", http.MethodGet,
		"/d/"+doc.Sqid+"/updates?since=1", nil))
	if snap.Code != http.StatusOK {
		t.Fatalf("GET = %d, want 200", snap.Code)
	}
	raw := serve(app, app.TestRequest(t, "owner@example.com", http.MethodGet,
		"/d/"+doc.Sqid+"/updates?since=1&raw=1", nil))

	if snap.Body.Len() >= raw.Body.Len() {
		t.Errorf("snapshot response %d bytes, raw log %d — the snapshot bought nothing",
			snap.Body.Len(), raw.Body.Len())
	}
	if got := docsSeq(t, snap.Result()); got != int64(snapshotCadence+3) {
		t.Errorf("Docs-Seq = %d, want %d", got, snapshotCadence+3)
	}
}

// raw=1 is the escape hatch a browser needs when a snapshot will not
// decode: doc_updates rows only, which is where the log's authority is
// actually reachable from.
func TestRawParameterReturnsOnlyLogRows(t *testing.T) {
	app, err := NewForTest(t)
	if err != nil {
		t.Fatalf("NewForTest: %v", err)
	}
	ctx := context.Background()
	doc := pastCadence(t, app, "Raw")

	raw := serve(app, app.TestRequest(t, "owner@example.com", http.MethodGet,
		"/d/"+doc.Sqid+"/updates?raw=1", nil))
	if raw.Code != http.StatusOK {
		t.Fatalf("GET raw = %d, want 200", raw.Code)
	}

	var rows []DocUpdate
	if err := app.Store().db.WithContext(ctx).
		Where("doc_id = ?", doc.ID).Order("seq ASC").Find(&rows).Error; err != nil {
		t.Fatalf("load log: %v", err)
	}
	var want []byte
	for _, r := range rows {
		want = append(want, r.UpdateBin...)
	}
	if !bytes.Equal(raw.Body.Bytes(), want) {
		t.Error("raw=1 did not return exactly the concatenated log")
	}
}

// The live-sync design's D2 survives snapshots: the header is still
// derived from the bytes in hand.
func TestDocsSeqStillDerivedWithSnapshots(t *testing.T) {
	app, err := NewForTest(t)
	if err != nil {
		t.Fatalf("NewForTest: %v", err)
	}
	doc := pastCadence(t, app, "Header")
	top := int64(snapshotCadence + 3)

	behind := serve(app, app.TestRequest(t, "owner@example.com", http.MethodGet,
		"/d/"+doc.Sqid+"/updates?since=2", nil))
	if got := docsSeq(t, behind.Result()); got != top {
		t.Errorf("Docs-Seq behind the snapshot = %d, want %d", got, top)
	}

	ahead := serve(app, app.TestRequest(t, "owner@example.com", http.MethodGet,
		"/d/"+doc.Sqid+"/updates?since="+strconv.FormatInt(top+5, 10), nil))
	if ahead.Body.Len() != 0 {
		t.Errorf("a cursor ahead of the log returned %d bytes, want none", ahead.Body.Len())
	}
	if got := docsSeq(t, ahead.Result()); got != top+5 {
		t.Errorf("Docs-Seq ahead of the log = %d, want %d (an echo of since)", got, top+5)
	}
}

• [ ] Step 2: Run and watch them fail.

• [ ] Step 3: Implement. In updatesGet: parse raw; when not raw, read newestSnapshot, and if it exists and snapshot.Seq > since, write the snapshot's State first and then rows with seq > snapshot.Seq; otherwise behave exactly as today. Docs-Seq remains max(since, highest seq in the bytes actually written) — the snapshot's own seq column counts, because it travels with bytes that contain its whole prefix. Update the handler's doc comment: it no longer returns the raw log.

• [ ] Step 4: Run the full suite. Commit.

---

Task 7: The browser can escape a corrupt snapshot

Files:

• Modify: fixtures/live-updates.mjs, fixtures/live-updates.test.mjs, fixtures/editor-entry.js
• Rebuild: internal/docs/static/editor.<hash>.js

• [ ] Step 1: Write the failing test

test('a decode failure retries once through the raw log, then gives up', async () => {
  const { live, stream, fetches, applied } = build({
    onUpdates: () => { throw new Error('bad update') },
  })
  live.start()
  stream.emit(1)

  fetches.calls[0].settle({ bytes: new Uint8Array([1]), seq: 1 })
  await Promise.resolve(); await Promise.resolve()

  assert.equal(fetches.calls.length, 2, 'no retry after a decode failure')
  assert.equal(fetches.calls[1].raw, true, 'the retry did not ask for the raw log')
  assert.equal(live.cursor, 0, 'the cursor advanced over bytes that never applied')
})

fakeFetches must record the raw option; extend it rather than adding a second fake.

• [ ] Step 2: Run and watch it fail.

• [ ] Step 3: Implement. fetchSince(since, { raw }); in drain, wrap the onUpdates call so a throw retries the same cursor once with raw: true, and a second throw reports through onError and leaves the cursor alone. Document why: a snapshot corrupted on disk after it was verified is invisible to the server, which serves the blob without a core call, so the browser is the only reader that can detect it and the only one that needs a way out.

• [ ] Step 4: Wire the same fallback into editor-entry.js's initial load, which fetches before any subscriber exists.

• [ ] Step 5: Rebuild the bundle — cd fixtures && node build-bundles.mjs — and commit the result. Do not hand-edit it.

• [ ] Step 6: Run node --test live-updates.test.mjs, then the gate. Commit.

---

Task 8: It actually works

• [ ] Step 1: .amadan/ci.d/10-gate, exit 0.
• [ ] Step 2: A document past the cadence still edits in a real browser. Drive the editor, type past a snapshot boundary, reload, and confirm the content survives — the one check that exercises snapshot, tail, and the client together.
• [ ] Step 3: Measure what this bought. Time a replay of a document with several thousand updates before and after. The number belongs in the branch discussion; it is the evidence the cadence is right.