Spec: live updates across tabs, over an SSE advisory
Edits reach every open editor of a document without a reload. The stream carries a seq number and nothing else; the client fetches the actual updates through the binary endpoint it already uses at mount, so document bytes keep one path and SSE's text-only framing costs no base64. Reviewed by Fable at the design stage per AGENTS.md, which found two defects now folded in. The first was silent data loss: deriving the Docs-Seq header from a second MAX(seq) query is a TOCTOU across the read/write pool split, and a client could advance its cursor over a row it never received. The header is now derived from the bytes in hand, which makes that interleaving unrepresentable. The second was narrower than it looks: closing an idle stream and reopening only on visibility, focus or a local edit leaves a tab that is visible and focused but merely being read permanently stale, which is the watching half of the whole feature. Section 9 records the one ruling left for Paul: what ships is not section 7.1's fallback as written (a per-connection counter), but a document-global log seq, so 7.1 wants amending rather than annotating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
docs/superpowers/specs/2026-09-01-live-sync-design.md+403 −0
Intent
<task-notification> <task-id>bhfxsiy7b</task-id> <tool-use-id>toolu_01GRokjACsQjPeQkMrSihtuA</tool-use-id> <output-file>/home/paulca/.cache/tmp/claude-1001/-home-paulca-amadan-net-oficina-docs/4a30c557-129a-4e57-8aa3-0c7518c36cf1/tasks/bhfxsiy7b.output</output-file> <status>completed</status> <summary>Background command "Install fixtures node deps" completed (exit code 0)</summary> </task-notification>
<task-notification> <task-id>bnk410t1x</task-id> <tool-use-id>toolu_01U22q22unGnbSKEw2qe9uoH</tool-use-id> <output-file>/home/paulca/.cache/tmp/claude-1001/-home-paulca-amadan-net-oficina-docs/4a30c557-129a-4e57-8aa3-0c7518c36cf1/tasks/bnk410t1x.output</output-file> <status>completed</status> <summary>Background command "Baseline Go test run" completed (exit code 0)</summary> </task-notification>
<task-notification> <task-id>bmqta3kz5</task-id> <tool-use-id>toolu_01V88aj6FmeRqMBRj4b2MXDs</tool-use-id> <output-file>/home/paulca/.cache/tmp/claude-1001/-home-paulca-amadan-net-oficina-docs/4a30c557-129a-4e57-8aa3-0c7518c36cf1/tasks/bmqta3kz5.output</output-file> <status>completed</status> <summary>Background command "Baseline Go tests with writable cache" completed (exit code 0)</summary> </task-notification>
<task-notification> <task-id>a7c87387c55b3bbdd</task-id> <tool-use-id>toolu_014zexCD1C74RZuFbutGCycM</tool-use-id> <output-file>/home/paulca/.cache/tmp/claude-1001/-home-paulca-amadan-net-oficina-docs/4a30c557-129a-4e57-8aa3-0c7518c36cf1/tasks/a7c87387c55b3bbdd.output</output-file> <status>completed</status> <summary>Agent "Fable review of live-sync design" finished</summary> <note>A task-notification fires each time this agent stops with no live background children of its own. The user can send it another message and resume it, so the same task-id may notify more than once.</note> <result>The review is complete — everything read, nothing in the repo touched. Here is the review. --- # Review: live-sync design (SSE advisory + POST), pre-spec **Verdict: approve with changes.** The transport ruling, the broadcast seam, the client module shape, and all three claims about existing code are correct. Two genuine defects need fixing before this becomes a spec — one silently loses an update (the exact failure class the design says it most fears), one makes the demo scenario fail for a visibly parked tab. The rest is material the spec should state, not things that are wrong. ## Genuinely wrong ### 1. SEVERE — the empty-slice `Docs-Seq` rule is a TOCTOU that skips a row permanently Design §Server-1 (live-sync-design.md:77–78): `Docs-Seq` is "the highest seq included, **or the document's current max when the slice is empty**." That second clause requires a separate `MAX(seq)` query after the slice query, and reads go to the read pool — nothing serializes them against the single-connection write pool's commits. Interleaving: tab A cursor 45, fetches `?since=45`. Handler's slice query runs → empty. Tab B's POST commits row 46 (store.go's transaction) and broadcasts. Handler's `MAX` query runs → 46. Response: empty body, `Docs-Seq: 46`. A advances its cursor to 46 **without ever applying row 46**. The broadcast that would have saved it arrived while A's fetch was in flight; the design's own fetch-serialization ("a full slot means a notification is already pending") coalesces it into a follow-up `?since=46` → empty. Row 46 is invisible in tab A until a full reload. This is exactly the "being wrong loses content silently" case §Server-2 promises its own test for — but the hole is in the GET header rule, not the cursor+1 rule. **Smallest fix:** `Docs-Seq = max(since, highest seq actually returned)` — never a second query. Empty slice echoes `since`; the no-`since` initial GET degenerates to 0 on an empty document, which still seeds the cursor correctly. Client side: cursor updates are always `cursor = max(cursor, header)`, never assignment, so a stale response can never regress it. Both belong in the promised both-sides test. ### 2. MODERATE — a visible, focused, idle tab goes stale with no path back Design §Server-5/Client-1 (live-sync-design.md:130–135): stream closes "after an idle period with no local edits and nothing received — reopening on **visibility, focus, or the next local edit**." A tab that is already visible and already focused while its person reads a long document fires none of those: the idle timer closes the stream, and nothing ever reopens it. Someone edits on their laptop; the desktop tab they are looking at never updates — the literal demo ("type in one, watch the other") fails whenever the watcher has been reading quietly past the idle threshold. Spec §7.3 says "**any activity** reconnects," which is broader than the design's three triggers; the design narrowed it and broke it. **Smallest fix:** count generic interaction — passive `pointermove`/`scroll`/`keydown` listeners — as the activity that both defers idle-close and reopens the stream. No timers, no polling (a slow poll would wake a hibernated instance and be worse than the stream), so the hibernation story is undamaged. ## Correct, but must be stated in the spec (silent otherwise) ### 3. `EventSource` native auto-reconnect must be disabled everywhere, not just for hidden tabs The design notes auto-reconnect "would defeat the TTL" only in the hidden/idle discussion. It defeats *every* policy decision: the browser retries on any server close, including the idle-close the module just chose. This is the one piece of jobs/events.go whose reason does not survive the move — jobs *relies* on native auto-reconnect ("EventSource does so on its own", events.go:16–18) because it has no close policy; this design must never rely on it. State: every close is `es.close()` by the module, every open is a deliberate decision, TTL closes reconnect only while visible-and-active (with jitter is nice, unnecessary at this scale). Miss this and a parked tab's browser reopens the stream forever — §7.3's exact failure. The rest of the recipe survives with its reasons intact: the write deadline (serve.go:275–286's zero-`WriteTimeout` contract, confirmed — `cmd/docs/main.go` goes through `rastrillo.Resolve` with defaults), the heartbeat (load-bearing here: it is what makes the write deadline ever fire on a quiet stream, bounding dead-client discovery to ~45s), the TTL as hard bound, and the hub replacing the 1s tick (the tick existed only because jobs had no broadcast primitive — its own comment says so). I checked the rest of rastrillo v0.22.0: no WebSocket, pubsub, or streaming helper anywhere else (`jobs/events.go` is the only `text/event-stream` producer; `internal/devloop` is unexported dev tooling). The design missed nothing there. ### 4. The committed-row-then-500 path drops a notification and the header `Store.ApplyUpdate` commits the `doc_updates` row, then bumps `updated_at` *outside* the transaction (store.go, the trailing `owned(...).Update`). If the bump fails, ApplyUpdate returns an error over a durable row: the handler 500s, no broadcast, no `Docs-Seq`; the client retries and writes a **duplicate row** (Yjs-idempotent, converges, but costs a `doc_updates` row and a `maxRetainedUpdates` slot), and only the retry broadcasts. Not a data-loss bug — the cursor+1 rule handles it (cursor stays put, next fetch catches up) — but the spec must define the return contract: seq is meaningful only when `err == nil`, broadcast only on nil error, and the duplicate-row-on-retry outcome is accepted. Same applies to a connection dropped after commit. ### 5. Small protocol clarifications the tests should pin - Stream events (`data: <seq>`) are **advisory only** — a fetch trigger, never a cursor assignment. The design implies it; say it. - A stopped `currentLive` instance must drop a late-resolving `?since=` response rather than deliver it (harmless today only because the callback closes over the discarded ydoc — make it a rule, not an accident). - A failed cursor+1 advance while the stream happens to be closed (debounce flush firing after the tab hid) should also mark "behind", healed by the connect-time max event on reopen — which is why that connect-time event is load-bearing and deserves its own test. ## Answers to the six questions **1. Transport — A is right, and "fallback first" is defensible.** B's cost claims all verified: no WS helper in rastrillo v0.22.0; `internal/core` ABI v2 exports no state-vector/diff entry points (an ABI bump D2 deliberately avoided); §7.3's hibernation problem is real. One correction for the amendment: §7.1's fallback describes "a monotonic sequence number **per connection**" with a high-water ack — the design's `Docs-Seq` is the document's global log seq, a different and better protocol. When Paul rules on the open question (correctly left as his call per AGENTS.md), §7.1 should be amended to describe the doc-seq protocol, not annotated as "the fallback shipped" — because what is shipping is not §7.1's fallback as written. **2. Cursor+1 — the rule is correct.** I constructed: two tabs interleaving (loser stays at its cursor, fetch applies both rows, its own row is a Yjs no-op); beacon racing flush *including* the beacon-fires-but-page-survives browser quirk (`pendingBytes()` doesn't clear the queue, so a duplicate row lands; no cursor movement since a beacon reads no response; idempotent); 5xx retry over a committed row (finding 4); two concurrent flushes from one tab with inverted response order (save-queue.mjs has that window — flush1 awaiting `post` while a new timer fires flush2; safe: Yjs parks missing-dep updates pending, and both rows are own bytes); resync mid-flight (old instance's late fetch hits the discarded ydoc). None loses, duplicates-with-effect, or wedges — a non-advanced cursor is always healed by the connect-time event or next fetch. The one silent-loss hole in the protocol is server-side finding 1, not this rule. **3. Broadcast seam — handler is right.** Store stays a persistence type, `doc_updates` stays §4.3's plain append-only table, and the only update writer is `updatesPost` (beacon and keepalive fetch both route through it, so both notify). The seed path (`seedIfEnabled` → `Seed` → `ApplyUpdate`) writes before any editor for that person can exist, so the accepted no-notify consequence has no reachable case today. Fold finding 4's ordering (broadcast on nil error only) into the seam's contract. **4. Hibernation — holds on the server end, holds on the client end only after findings 2 and 3.** Server: no timer exists outside a live connection, TTL is the hard bound, heartbeat+deadline reap wedged clients. A visible active tab reconnecting every 5 minutes pins the instance — by design, per §7.3 (someone is there). The permanently-stale-tab case is finding 2; the pinned-awake case is finding 3's native auto-reconnect. **5. Copied recipe — covered under finding 3.** One thing the dropped tick did that the hub does not: re-check the entity's existence each second (jobs' `gone` event). No delete route exists yet and TTL bounds exposure to 5 minutes; leave a note for the day deletion lands. **6. Yjs semantics — nothing wrong, one omission worth a line.** `applyServerLog`'s shared-decoder `Y.readUpdate` loop is exactly right for the concatenated slice (same framing as the full log). Remote updates onto a doc with local edits: ySyncPlugin mirrors local PM transactions into the Y.Doc synchronously (editor-entry.js's own comment), so there is no uncommitted CRDT state — merging is pure Yjs. Out-of-order and duplicate updates are genuinely safe (idempotent integration; missing-dep updates park in the doc's pending set; the server log is causally adequate since the core applied each row against full replayed history). The caret claim is true — ySyncPlugin maps selection through relative positions across remote transactions. The undo trap: the design never mentions `yUndoPlugin`, and it should — it does the right thing *by default* (its `Y.UndoManager` tracks only the ySyncPlugin's own origin, so `'server'`-origin updates are never captured and Mod-z after a remote edit undoes your own edit, not theirs), and one line in the spec prevents an implementer from "helpfully" adding `'server'` to `trackedOrigins`, which would break it. Two cosmetic notes for the spec: a remote transaction landing mid-IME-composition can cancel composition (known y-prosemirror behavior, not data loss), and undo history resets on resync (whole doc rebuilt — existing behavior). ## The three code claims — all three verified true - **`ApplyUpdate` returning seq:** true. Seq is `maxSeq+1` inside the `Transaction` closure (store.go); hoisting it to a captured variable and returning after commit disturbs neither the single-writer MAX+INSERT guarantee nor the owner re-derivation, which runs before. Contract caveat in finding 4. - **Save queue unchanged:** true. `post: (bytes) => fetch(...)` in editor-entry.js's `makeSaveQueue` can become an async closure that awaits the fetch, reads `Docs-Seq` **on 2xx only**, and returns the response; save-queue.mjs consumes only `r.ok`/`r.status`. The beforeunload beacon/keepalive path bypasses `post` and reads no header — harmless, analyzed under question 2. - **Echo-loop prevention:** true. `ydoc.on('update')` returns on `origin === 'server'`; `Y.readUpdate(decoder, ydoc, 'server')` tags every server byte; and ySyncPlugin does not re-emit remote-origin changes as new Yjs updates, so there is no ProseMirror-level echo either. Minor: the debounce recommendation (keep 2000ms, define "live" honestly) is right and matches core.go's own `maxRetainedUpdates` arithmetic; the testing section is sound — add finding 1's header rule and cursor monotonicity to it. Key files: `/home/paulca/.claude/jobs/4a30c557/tmp/live-sync-design.md` (under review), `/home/paulca/amadan.net/oficina/docs/internal/docs/store.go` (ApplyUpdate transaction), `/home/paulca/amadan.net/oficina/docs/internal/docs/handlers.go` (updatesGet — where finding 1's fix lands), `/home/paulca/amadan.net/oficina/docs/fixtures/editor-entry.js` and `save-queue.mjs` (claims B/C), `/home/paulca/go/pkg/mod/github.com/carlosframework/rastrillo@v0.22.0/jobs/events.go` (the recipe).</result> <usage><subagent_tokens>120203</subagent_tokens><tool_uses>26</tool_uses><duration_ms>376085</duration_ms></usage> </task-notification>
diff --git a/docs/superpowers/specs/2026-09-01-live-sync-design.md b/docs/superpowers/specs/2026-09-01-live-sync-design.md| new file mode 100644 |
| index 0000000..d80821c |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-09-01-live-sync-design.md |
| @@ -0,0 +1,403 @@ |
| +# Live Updates Across Tabs — Design |
| + |
| +**Status:** designed, reviewed by Fable (2026-09-01), one ruling open for Paul (§9). |
| +**Spec it extends:** `docs/superpowers/specs/2026-08-30-docs-design.md` — §4 (data |
| +model), §4.1 (the log is bounded), §4.3 (rejected `eventlog`), §7.1 (merge and |
| +transport), §7.3 (presence and hibernation). |
| +**Plan it produces:** `docs/superpowers/plans/2026-09-01-live-sync.md`. |
| + |
| +## 1. What this is |
| + |
| +Open a document in two tabs, type in one, and the other updates without a reload. |
| +Same person, two tabs, two browsers, or two devices — anywhere a document you own is |
| +open. |
| + |
| +This is the content half of §7's collaboration story, and only that half. It is the |
| +"later plan" the editing plan's D1 promised: *"Live collaboration, presence, and the |
| +socket's idle timeout arrive together in a later plan, and the update-log format this |
| +plan establishes is what that plan will stream."* Presence and the socket do not |
| +arrive here — see §8 for why they are separable, and §10 for what is deliberately |
| +left out. |
| + |
| +## 2. Where the code is today |
| + |
| +Editing is plain HTTP. The browser fetches `GET /d/{sqid}/updates` |
| +(`internal/docs/handlers.go:311`), which returns every row of the document's update |
| +log concatenated in seq order, and replays it into a `Y.Doc`. Edits produce Yjs |
| +updates which are debounced 2000ms, merged, and POSTed back |
| +(`internal/docs/handlers.go:359`), validated through the Rust core and appended to |
| +`doc_updates`. Both routes are owner-scoped through one helper |
| +(`getDocumentForUpdates`, `handlers.go:277`) and sit inside the `RequireSession` group |
| +under `csrf.Protect` (`app.go:435–455`). |
| + |
| +Nothing pushes to an open editor. A second tab sees changes only on reload. That is |
| +the whole gap this design closes. |
| + |
| +Two existing mechanisms this design leans on rather than replaces: |
| + |
| +- **`doc_updates.seq`** is allocated MAX+INSERT inside a transaction |
| + (`store.go:539`). The long comment above it explains why that is safe rather than |
| + racy: rastrillo/db caps the write pool at exactly one connection, so a second |
| + concurrent `ApplyUpdate` cannot begin its transaction until this one commits. |
| +- **The client's resync path** (`attemptResync` in `fixtures/editor-entry.js`, driven |
| + by `resync-controller.mjs`) already rebuilds the `Y.Doc`, the save queue and the |
| + `EditorView` wholesale from the server's log. Live updates hang off that same |
| + lifecycle rather than inventing a second one. |
| + |
| +## 3. Transport: SSE advisory, not a WebSocket |
| + |
| +§7.1 specifies a WebSocket with an SSE fallback. This ships an SSE-shaped transport |
| +first, deliberately. |
| + |
| +**What a WebSocket would cost here.** rastrillo v0.22.0 has no WebSocket helper — a |
| +new dependency in a deliberately lean `go.mod`. The y-protocols sync handshake would |
| +be hand-rolled in Go, needing state-vector and diff entry points the Rust core does |
| +not export; that is an ABI bump, which is exactly the work the editing plan's D2 |
| +avoided by having the browser replay raw update bytes. `y-websocket` would need |
| +vendoring into a bundle this app builds by hand. `Origin` would need checking |
| +manually, since AGENTS.md's `Sec-Fetch-Site` reasoning covers ordinary requests and |
| +not a WebSocket handshake. And §7.3's hibernation problem would get worse rather than |
| +deferred. Most of that expense buys awareness, which §8 removes from this slice. |
| + |
| +**What this costs instead.** One extra round trip per change, and a transport that is |
| +not yet what §7.1 describes. |
| + |
| +**Rejected alternatives.** Polling `?since=` on a timer: simplest, but "live" becomes |
| +seconds late and a parked tab polls forever, which pins a hibernating instance harder |
| +than a stream that can close. A same-browser `BroadcastChannel`: nearly free, but only |
| +syncs one browser's own tabs, which is not what was asked. |
| + |
| +### 3.1 The stream carries no document bytes |
| + |
| +`jobs/events.go` in rastrillo — the only `text/event-stream` producer in the |
| +framework — carries no HTML, on the reasoning that keeping one rendering path is worth |
| +more than saving a round trip. The same reasoning applies to bytes: the stream carries |
| +a seq number, and the client fetches the actual updates through the same binary |
| +endpoint it already uses at mount, through the same `applyServerLog`. |
| + |
| +This also sidesteps SSE being a text protocol. Yjs updates are opaque binary; putting |
| +them on the stream would mean base64 and roughly 33% overhead on every keystroke |
| +batch, plus a second code path for document bytes that would need its own tests and |
| +could drift from the first. |
| + |
| +## 4. Protocol |
| + |
| +### D1 — `seq` becomes a wire value |
| + |
| +`doc_updates.seq` already exists and is already allocated safely. This promotes it |
| +from an internal ordering column to part of the client-facing contract. |
| + |
| +- `Store.ApplyUpdate` returns `(int64, error)` — the assigned seq — instead of `error`. |
| + **The seq is meaningful only when `err == nil`.** |
| +- `GET /d/{sqid}/updates` accepts an optional `?since=N` and returns only rows with |
| + `seq > N`. `maxLogBytes` still applies to the slice, still checked before the first |
| + byte is written. |
| +- Both `GET` and `POST /d/{sqid}/updates` carry a `Docs-Seq` response header. |
| +- A malformed `since` (non-numeric, negative) is a 400. |
| + |
| +### D2 — `Docs-Seq` is derived, never queried |
| + |
| +**`Docs-Seq` = `max(since, highest seq in the bytes actually returned)`.** |
| + |
| +An earlier draft said "the highest seq included, or the document's current max when |
| +the slice is empty". Fable's review found that this loses an update, silently and |
| +permanently. The second clause needs a separate `MAX(seq)` query, and reads go to the |
| +read replicas while writes go to the single-connection write pool — nothing serializes |
| +the two: |
| + |
| +> Tab A is at cursor 45 and fetches `?since=45`. The slice query returns empty. Tab B's |
| +> POST commits row 46 and broadcasts. A's `MAX(seq)` query then runs and returns 46. |
| +> A receives an empty body and `Docs-Seq: 46`, and advances its cursor to 46 having |
| +> never applied row 46. The broadcast that would have rescued it arrived while A's |
| +> fetch was in flight and was coalesced into a follow-up `?since=46` — which is also |
| +> empty. Row 46 stays invisible in tab A until a manual reload. |
| + |
| +Deriving the header from the bytes in hand makes that interleaving unrepresentable: |
| +an empty slice echoes `since` back, so a cursor never moves over a row the client did |
| +not receive. The no-`since` initial GET on an empty document derives 0, which seeds |
| +the cursor correctly. |
| + |
| +**The client half of the same rule: a cursor only ever moves forward.** |
| +`cursor = max(cursor, header)`, never assignment, so a response that arrives out of |
| +order cannot drag a cursor backwards. |
| + |
| +### D3 — the cursor+1 rule for a tab's own POST |
| + |
| +A tab that POSTs and gets `Docs-Seq: 43` back may advance its cursor to 43 **only if |
| +its cursor was 42** — only if `seq == cursor + 1`. |
| + |
| +Otherwise another tab wrote rows in between that this tab has not applied, and |
| +adopting 43 would skip them permanently. When the rule does not hold the cursor stays |
| +put, the tab is "behind", and an ordinary `?since=` fetch — triggered by the stream, |
| +or by the connect-time event on the next reopen — heals it. |
| + |
| +The rule earns its place by removing the echo round trip: in the common case (one tab |
| +typing) every POST advances the cursor without a fetch, and the broadcast for that seq |
| +is a no-op when it arrives. |
| + |
| +Fable constructed the adversarial interleavings — two tabs racing, a `beforeunload` |
| +beacon racing a flush including the case where the beacon fires but the page survives, |
| +a 5xx retry over an already-committed row, two concurrent flushes from one tab with |
| +responses arriving inverted, and a resync mid-flight — and none of them loses an |
| +update, duplicates one with effect, or wedges a cursor permanently. A cursor that |
| +fails to advance is always healed; that is the property to preserve if this rule is |
| +ever changed. |
| + |
| +### D4 — broadcast on success only |
| + |
| +The handler broadcasts after `ApplyUpdate` returns a nil error, never before and never |
| +on an error. |
| + |
| +This matters because `ApplyUpdate` commits the `doc_updates` row in its transaction |
| +and then bumps `documents.updated_at` *outside* it. If that bump fails, the method |
| +returns an error over a row that is already durable. The handler 500s, sends no |
| +`Docs-Seq`, and broadcasts nothing; the client's save queue treats a 5xx as retryable |
| +and re-POSTs the same bytes, which lands a **duplicate row**. That is accepted: Yjs |
| +integration is idempotent so the document converges, the cursor+1 rule keeps the |
| +client honest (its cursor never moved), and the cost is one wasted row against |
| +`maxRetainedUpdates`. The same is true of a connection dropped after commit. |
| + |
| +The alternative — broadcasting from inside `Store.ApplyUpdate` — is rejected in D5. |
| + |
| +### D5 — the broadcast seam is the handler, not `Store` |
| + |
| +`Store` stays a persistence type that knows nothing about transports, and |
| +`doc_updates` stays "a plain append-only table" as §4.3 insisted when it rejected |
| +`eventlog` for carrying JSON payloads and a redundant Lamport order. |
| + |
| +The accepted consequence: a writer that does not go through `updatesPost` does not |
| +notify. Today the only other writer is sample seeding (`seedIfEnabled` → `Seed` → |
| +`ApplyUpdate`), which runs when a person's row is first created — before any editor of |
| +theirs can be open — so the consequence has no reachable case. A future importer would |
| +need to notify explicitly, and this paragraph is the note that says so. |
| + |
| +## 5. Server design |
| + |
| +### 5.1 The hub — `internal/docs/stream.go` (new) |
| + |
| +A `map[int64]map[chan int64]struct{}` behind a mutex: subscribe by document id and get |
| +a one-slot channel, unsubscribe on stream close. Sends are non-blocking — a full slot |
| +means a notification is already pending, and since every client fetch is |
| +`since=cursor`, coalescing a burst into one wake-up loses nothing. The seq on the |
| +channel is advisory: it tells a client *that* there is something new, and the fetch |
| +determines *what*. |
| + |
| +One instance per team, one process — the docs spec's opening paragraph ("One instance |
| +per team") and its §3, System shape ("One SQLite file per instance") — so there is |
| +nothing to coordinate across machines and no shared-broker-shaped hole here. |
| + |
| +### 5.2 The stream — `GET /d/{sqid}/stream` |
| + |
| +Owner-scoped through the existing `getDocumentForUpdates` helper, inheriting its |
| +404-not-403 behaviour unchanged (a stranger must not learn the document exists). |
| +Registered in the same `RequireSession` group as the update routes; a GET, so |
| +`csrf.Protect` passes it through untouched. |
| + |
| +The body follows `jobs/events.go`: an `http.Flusher` check, `http.NewResponseController`, |
| +`text/event-stream` with `Cache-Control: no-store`, a 200 and a flush before the first |
| +event, a 30-second write deadline re-armed before every write, a `: ping` comment |
| +heartbeat every 15 seconds, a 5-minute TTL, and a `select` that includes |
| +`r.Context().Done()` so a graceful drain is never held open. |
| + |
| +Each piece of that recipe is carried for its own reason, not by imitation: |
| + |
| +- **The write deadline** is what lets a stream coexist with `serve`'s zero |
| + `WriteTimeout` default (`serve.go`'s own Timeouts recipe). `cmd/docs/main.go` goes |
| + through `rastrillo.Resolve` with the defaults, so that zero applies here. |
| +- **The heartbeat** is what makes the write deadline ever fire on a quiet stream. It |
| + is what bounds discovery of a dead client to about 45 seconds instead of never. |
| +- **The TTL** is the hard bound on any one connection, and half of the hibernation |
| + story (§6). |
| +- **The 1-second database tick is dropped.** `jobs/events.go` polls because, in its own |
| + words, it had no broadcast primitive to wait on. This has one, so the handler blocks |
| + on the hub channel and never polls. One thing the tick also did is not replaced: it |
| + re-checked the entity's existence every second and could emit `gone`. There is no |
| + delete route yet, and the TTL bounds the exposure to five minutes; when deletion |
| + lands, this needs revisiting. |
| + |
| +Events are `event: update` with `data: <seq>`. **One is sent immediately on connect, |
| +carrying the highest seq the server currently has.** That event is load-bearing rather |
| +than an optimisation: it is what heals a tab whose cursor legitimately failed to |
| +advance (D3), and what covers the gap between the page rendering and the stream |
| +opening. It gets its own test. |
| + |
| +### 5.3 Deliberately omitted: a subscriber cap |
| + |
| +Every route here is behind `RequireSession`, a document has exactly one owner today, |
| +and a subscriber costs a goroutine and a one-slot channel. Revisit when sharing lands |
| +and a document can have many viewers. |
| + |
| +## 6. Hibernation (§7.3) |
| + |
| +§7.3's requirement: *"The editor socket has an idle timeout: no edits and no presence |
| +activity for a set period closes it, and any activity reconnects. A parked tab stops |
| +holding the instance open."* |
| + |
| +**Server.** The 5-minute TTL is the hard bound. No timer exists outside a live |
| +connection, so nothing here is a `time.Ticker` that dies at hibernation — the trap |
| +§4.1 and §7.3 both name. |
| + |
| +**Client.** Three rules, and the first is the one an implementer will get wrong: |
| + |
| +1. **The module owns every open and every close.** `EventSource` reconnects natively |
| + on any server close, which would defeat every policy decision below — including the |
| + idle-close the module just made. So native reconnection is never relied on: every |
| + close is an explicit `es.close()`, and every open is a deliberate decision. This is |
| + the one piece of `jobs/events.go`'s recipe whose reason does *not* survive the move: |
| + that handler relies on native auto-reconnect precisely because it has no close |
| + policy of its own. Miss this and a parked tab's browser reopens the stream forever, |
| + which is §7.3's exact failure. |
| +2. **Close when hidden, or when idle.** A hidden tab closes immediately; a visible one |
| + closes after an idle period with no local edits and nothing received. |
| +3. **"Any activity" means any activity.** Visibility, focus and local edits are not |
| + enough: a tab that is already visible and already focused while its person *reads* a |
| + long document fires none of them, so the idle timer would close the stream and |
| + nothing would ever reopen it — and the watching half of "type in one, watch the |
| + other" would go permanently stale. Passive `pointermove`, `scroll` and `keydown` |
| + listeners count as activity, both deferring the idle close and reopening a closed |
| + stream. |
| + |
| +A visible, active tab reconnecting every five minutes does hold its instance awake. |
| +That is intended: someone is there. |
| + |
| +## 7. Client design |
| + |
| +### 7.1 A new module, `fixtures/live-updates.mjs` |
| + |
| +Follows the convention `save-queue.mjs` and `resync-controller.mjs` set — dependencies |
| +injected, so it runs under plain Node with no DOM, no `EventSource` and no real |
| +`Y.Doc`, and its own `.test.mjs` goes in the gate. It owns: |
| + |
| +- the cursor, and D2's monotonic rule; |
| +- the `EventSource` lifecycle and every policy in §6; |
| +- fetch serialization: one in-flight `?since=` request at a time, with a pending flag, |
| + so a burst of notifications cannot interleave two applies; |
| +- delivering fetched bytes to its caller. |
| + |
| +Two rules its tests pin, because both are currently true by accident rather than by |
| +construction: a stopped instance drops a late-resolving fetch rather than delivering |
| +it, and a stream event is a fetch trigger, never a cursor assignment. |
| + |
| +### 7.2 Wiring, confined to `attemptResync` |
| + |
| +`attemptResync` already rebuilds the `Y.Doc`, save queue and `EditorView` wholesale, and |
| +is used both for the first mount and for every recovery. Live updates join that |
| +lifecycle: a `currentLive` alongside `currentQueue`/`currentView`, stopped and replaced |
| +exactly as those are. The initial GET reads `Docs-Seq` to seed the cursor. Fetched |
| +bytes go through the existing `applyServerLog`. |
| + |
| +Nothing in `save-queue.mjs` changes. The `post` closure that `editor-entry.js` already |
| +supplies becomes an async closure that reads `Docs-Seq` on a 2xx and returns the |
| +response unchanged; the queue consumes only `r.ok` and `r.status`. The |
| +`beforeunload` beacon bypasses `post` and reads no response at all, which is harmless: |
| +it moves no cursor. |
| + |
| +### 7.3 Echo, divergence, and undo |
| + |
| +**Echo** needs no new mechanism. `applyServerLog` already tags replayed bytes with |
| +`Y.readUpdate(decoder, ydoc, 'server')`, and the `ydoc.on('update')` handler already |
| +returns early on `origin === 'server'`. ySyncPlugin does not re-emit remote-origin |
| +changes as new Yjs updates either, so there is no ProseMirror-level echo. |
| + |
| +**Divergence.** While `resyncController.diverged` is true the local document is known |
| +poisoned and a rebuild is coming, so the subscriber stops applying. `attemptResync` |
| +then replaces it wholesale. No new state machine. |
| + |
| +**Undo.** `yUndoPlugin` already does the right thing: its `Y.UndoManager` tracks only |
| +ySyncPlugin's own origin, so `'server'`-origin updates are never captured and Mod-z |
| +after a remote edit undoes your own last edit rather than someone else's. |
| +**Do not add `'server'` to `trackedOrigins`.** This sentence exists because that |
| +change looks like a fix and is a bug. |
| + |
| +Two known cosmetic behaviours, neither data loss: a remote transaction landing |
| +mid-IME-composition can cancel the composition, and undo history resets on a resync |
| +because the whole document is rebuilt. |
| + |
| +### 7.4 A dead stream is a freshness failure, not a save failure |
| + |
| +`[docs-save-status]` means "your edit did not save". A stream failure must never light |
| +it: POST is unaffected and edits are still saving. Repeated `EventSource` failures stop |
| +the stream and leave the tab to heal on the next activity-triggered reopen. Logged to |
| +the console; nothing user-visible in this slice, and therefore no new catalog keys. |
| + |
| +### 7.5 The debounce stays at 2000ms |
| + |
| +`DEBOUNCE_MS` sets the floor on how live this can feel: the other tab cannot see an |
| +edit before the originating tab sends it. Shortening it spends headroom against |
| +`internal/core/core.go`'s `maxRetainedUpdates` (4096, no compaction yet) — 4096 × 2s is |
| +about 2h16m of continuous typing, 4096 × 0.8s about 55 minutes — and a sibling |
| +session's formatting toolbar makes the same arithmetic worse, since one toolbar command |
| +is one transaction is one retained update. |
| + |
| +So **"live" here means "within about two seconds of the typist pausing"**, stated |
| +plainly rather than papered over. Snapshot cadence and compaction (§4.1) are the real |
| +prerequisite for making it snappier, and they are their own plan. |
| + |
| +## 8. Why presence is separable |
| + |
| +§7.3 ties presence to the socket, and this design has no socket. That is not a |
| +deferral of something half-built: presence needs ephemeral, high-frequency, |
| +upstream-and-downstream messages (awareness), which is exactly what a WebSocket is for |
| +and exactly what an SSE-plus-POST pair is bad at. Building presence on this transport |
| +would mean building the wrong thing twice. |
| + |
| +It is also unreachable today. Documents are owner-scoped — `internal/docs/models.go` |
| +has `Document`, `Person` and `DocUpdate` and no `grants` table — so the only "other |
| +viewer" is the same person's other tab, and a presence stack that shows you yourself |
| +is worse than nothing. Presence arrives with sharing, on a socket, as §7 always |
| +intended. |
| + |
| +## 9. Open ruling for Paul |
| + |
| +**§7.1 should be amended rather than annotated, and the wording matters.** |
| + |
| +§7.1's fallback describes "the client POSTs batched updates with a monotonic sequence |
| +number **per connection**, and the server acknowledges the high-water mark". What this |
| +design ships is not that: `Docs-Seq` is the document's global log seq, durable in |
| +`doc_updates`, meaningful across connections and across tabs. It is a different |
| +protocol and a better one — a per-connection counter cannot heal a tab that reconnects, |
| +which is most of what §6 needs. |
| + |
| +So "the fallback shipped first" would be a false description of the code. The honest |
| +options are to amend §7.1 to describe the doc-seq protocol as the transport, with the |
| +WebSocket as the upgrade that arrives with presence — which is what this design |
| +assumes — or to rule otherwise. **Recorded as Paul's call; the branch does not merge |
| +until he makes it.** |
| + |
| +## 10. Non-goals |
| + |
| +Presence, remote cursors, sharing and grants, WebSocket transport, offline |
| +(`y-indexeddb`), snapshots and compaction. Each belongs to a spec section that already |
| +describes it; none is blocked by anything here. |
| + |
| +## 11. Testing |
| + |
| +- **Go, `internal/docs/stream_test.go` (new).** `httptest.NewRecorder` cannot express a |
| + long-lived stream — it cannot observe bytes before the handler returns — so these use |
| + `httptest.NewServer` and read the body incrementally, with short injected durations |
| + (mirroring `jobs/events.go`'s injectable `tick`/`ttl`/`heartbeat`). Cases: a POST from |
| + another request reaches a subscriber as `event: update` with the right seq; the |
| + connect-time event carries the current max; a stranger's document 404s; the TTL closes |
| + cleanly; a heartbeat appears on a quiet stream; a cancelled request context ends the |
| + handler. |
| +- **Go, `internal/docs/updates_test.go` (extend).** `?since=` slicing; `Docs-Seq` on both |
| + GET and POST; **D2's derivation — an empty slice echoes `since` and never reports a |
| + higher number**; a malformed `since` is a 400; `maxLogBytes` still enforced on a slice. |
| +- **Node, `fixtures/live-updates.test.mjs` (new).** D3 including the case where the |
| + cursor must *not* advance; D2's monotonic cursor under an out-of-order response; |
| + fetch serialization under a burst; notification coalescing; the §6 close/reopen |
| + policy including the visible-and-idle case; a stopped instance dropping a late fetch. |
| +- **Acceptance.** `./scripts/run-dev.sh`, one document open in two tabs, type in one, |
| + watch the other. That is the ask, and it belongs in the plan as a step. |
| +- `.amadan/ci.d/10-gate` green, with the rebuilt editor bundle committed. |
| + |
| +## 12. Review record |
| + |
| +Reviewed by **Fable** on 2026-09-01, per AGENTS.md's highest-available-capability rule, |
| +at the design stage rather than after implementation. Verdict: approve with changes. |
| +Two defects found and folded in above — the `Docs-Seq` TOCTOU (D2) and the |
| +visible-idle-tab staleness (§6.3) — along with the `EventSource` auto-reconnect rule |
| +(§6.1), the broadcast-on-success contract (D4), the `trackedOrigins` warning (§7.3), and |
| +the §7.1 wording correction (§9). The review verified all three of the design's claims |
| +about existing code, and confirmed rastrillo v0.22.0 offers no WebSocket, pubsub or |
| +streaming helper beyond `jobs/events.go`. |