oficina / meet Public

Merge branch 'fix/screen-stall-escalation' into main

Screen stalls keyframes cannot mend (spec 15.22) and Reconnecting only
for a problem that lasts (15.23). Calls 0.4.5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude agent, operated by paul@keymail.dev 84ed5dc57e97d66dd57a67fddeff99102cc805d4 parent e7985a8parent 6b5a81f
31 files changed, +2782 −117
  • Makefile +1 −0
  • docs/superpowers/specs/2026-09-07-meet-design.md +131 −1
  • internal/calls/static/call-window.js +2 −0
  • internal/calls/static/calls.css +3 −0
  • internal/calls/static/room.js +563 −24
  • internal/calls/templates/room.html +6 −0
  • internal/callstest/native_browser_test.go +258 −11
  • native/README.md +37 −0
  • native/apple/Sources/CallsInterop/Interop.swift +41 −1
  • native/apple/Sources/CallsInterop/SyntheticVideo.swift +5 −1
  • native/apple/Sources/CallsKit/BroadcastScreenCapture.swift +73 −11
  • native/apple/Sources/CallsKit/Call.swift +156 −20
  • native/apple/Sources/CallsKit/CallAudio.swift +3 −1
  • native/apple/Sources/CallsKit/Capture.swift +7 −0
  • native/apple/Sources/CallsKit/ChatView.swift +1 −1
  • native/apple/Sources/CallsKit/DecoderWedge.swift +123 −0
  • native/apple/Sources/CallsKit/MacFullWindowCall.swift +1 −1
  • native/apple/Sources/CallsKit/MacScreenCapture.swift +3 −0
  • native/apple/Sources/CallsKit/Media.swift +55 −11
  • native/apple/Sources/CallsKit/ReconnectWatch.swift +125 −0
  • native/apple/Sources/CallsKit/ScreenWatchdog.swift +299 −11
  • native/apple/Sources/CallsMac/Resources/MacIcon.png binary
  • native/apple/Tests/CallsKitTests/ReconnectWatchTests.swift +102 −0
  • native/apple/Tests/CallsKitTests/ScreenWatchdogTests.swift +209 −6
  • native/ios/Sources/PhoneFullScreenCall.swift +1 −1
  • native/ios/project.yml +1 −1
  • test/browser/call.mjs +23 −2
  • test/browser/native-screen.mjs +332 −5
  • test/browser/room-reconnect.test.mjs +105 −0
  • test/browser/room-screen.test.mjs +111 −6
  • test/browser/room-share.test.mjs +5 −2

CI passed — run details

CI log
=== vet ===
go vet ./...
=== fmt ===
=== test ===
go test ./...
ok  	amadan.net/oficina/calls/callscore	(cached)
?   	amadan.net/oficina/calls/cmd/calls	[no test files]
ok  	amadan.net/oficina/calls/internal/calls	1.427s
ok  	amadan.net/oficina/calls/internal/calls/apns	(cached)
ok  	amadan.net/oficina/calls/internal/calls/hub	(cached)
?   	amadan.net/oficina/calls/internal/calls/icons	[no test files]
ok  	amadan.net/oficina/calls/internal/calls/relay	(cached)
ok  	amadan.net/oficina/calls/internal/calls/seal	(cached)
ok  	amadan.net/oficina/calls/internal/callstest	8.442s
?   	amadan.net/oficina/calls/protocol	[no test files]
?   	amadan.net/oficina/calls/scripts/writeestilo	[no test files]
=== migration-check ===
go run amadan.net/rastrillo/rastrillo/cmd/rastrillo migration check
rastrillo migration check: models and migrations agree
=== race ===
CGO_ENABLED=1 go test -race ./internal/...
ok  	amadan.net/oficina/calls/internal/calls	8.679s
ok  	amadan.net/oficina/calls/internal/calls/apns	(cached)
ok  	amadan.net/oficina/calls/internal/calls/hub	(cached)
?   	amadan.net/oficina/calls/internal/calls/icons	[no test files]
ok  	amadan.net/oficina/calls/internal/calls/relay	(cached)
ok  	amadan.net/oficina/calls/internal/calls/seal	(cached)
ok  	amadan.net/oficina/calls/internal/callstest	23.758s
diff --git a/Makefile b/Makefile
index 7c5b800..0075e0f 100644
--- a/Makefile
+++ b/Makefile
@@ -151,6 +151,7 @@ browser:
node --test test/browser/room-space.test.mjs
node --test test/browser/room-hearing.test.mjs
node --test test/browser/room-screen.test.mjs
+ node --test test/browser/room-reconnect.test.mjs
node --test test/browser/room-glow.test.mjs
node --test test/browser/room-layout.test.mjs
node --test test/browser/room-window.test.mjs
diff --git a/docs/superpowers/specs/2026-09-07-meet-design.md b/docs/superpowers/specs/2026-09-07-meet-design.md
index 58989e6..243ec8e 100644
--- a/docs/superpowers/specs/2026-09-07-meet-design.md
+++ b/docs/superpowers/specs/2026-09-07-meet-design.md
@@ -2415,7 +2415,7 @@ own screen, and the receiver says when a screen has stopped for it.
decoded nothing new here for four seconds while they say they are
sharing. Every sender repeats a still screen at least once a second, so
that is not a still screen. It is bad news only, omitted when there is
- none, held for at most a minute from the last frame, at most fifteen
+ none, held for at most a minute from the last frame (three since §15.22), at most fifteen
distinct well-formed peer ids, validated exactly as `noSoundFrom` is, and
re-encoded after it (`muted`, `cameraOff`, `audioBlocked`, `noSoundFrom`,
`screenStalled`). The raw payload cap is 2 KB, room for both lists full.
@@ -2436,3 +2436,133 @@ mute is never lost to an old server. The sharer's own watchdog works
whatever the server is. Deploy the server before, or with, the clients.
**Not logged, not stored.** As §15.19.
+
+### 15.22 §15.21 — when keyframes are not enough (2026-09-30)
+
+**Why.** Paul, 2026-09-30, sharing from the Mac app (0.4.3 or 0.4.4) to
+Samuel: "Still seeing issues with screen sharing after a while." His
+screenshot had §15.21's notice, "Samuel's view of your screen stalled —
+recovering", under "You are sharing your screen". The receiver's report had
+worked; recovery had not. By design, a report alone only ever bought
+keyframes, and a stall that keyframes do not mend — a receiver whose decoder
+has wedged, or one that has stopped taking this stream — had nothing after
+them: the notice sat at "recovering" until the report lapsed a minute later.
+Stop and Share, by hand, always mended it. Two more things came up reading the
+code: a receiver counted only a *larger* decoded count as movement, so a
+screen that had been given a new transceiver (whose count starts again)
+stayed "stalled" there for as long as its new count took to pass the old one;
+and the iPhone's broadcast sends nothing for a still screen, which any
+receiver reported as a stall after four seconds.
+
+**What the harness measured.** A receiver whose screen decoder is wedged
+(the gate's decoder wrapper in the apps, an encoded transform in Chromium) keeps
+receiving packets and frames, decodes none, and drops them: packets +75,
+frames received +32, decoded +0, dropped +29 in the first four seconds, between
+two apps. Keyframes arrive and are dropped with the rest. A new transceiver,
+which gets a new receiver and decoder at the far end, mends it at once, in both
+directions and between two apps. No long share has yet produced a stall that
+keyframes did not mend, so the wedge is a stand-in for the cause, not the cause:
+
+- Samuel was on the web, so the path was the Mac app to Chromium, where
+ §15.21 found H.264 held to level 3.1. With the level cap switched off
+ (`CALLS_SCREEN_UNCAPPED=1`), a Motion share of a 1920×1246 page (a 16:10
+ Retina display as the capture fits it) reproduces Paul's picture exactly:
+ VideoToolbox refuses every frame (-12902, 1,323 times in 150 s), the browser
+ receives nothing and sends PLIs, freezes of 7–23 s and one that never ended,
+ which keyframes, fresh encoders and a re-add only partly mend. With the cap as
+ shipped, the same page starting in Motion and starting in Balanced, modes
+ cycled every 30 s and the uplink capped from 600 kbit/s to 3 Mbit/s, eleven
+ ICE restarts from the app and twenty-three at the browser in four minutes
+ each: no -12902 and no freeze. The real display through ScreenCaptureKit
+ (3024×1964, sent at 1920×1246) for seven minutes the same way: no -12902; the
+ only gaps, 4 s each, were Text and detail on a 600 kbit/s link, where a
+ full-size frame of a still screen takes that long to send, which is the link's
+ and which the ladder does not take further (nothing was sent).
+ A screencast source (Balanced, Text and detail) is set up by this WebRTC
+ build without a fixed level and carries 1920×1246 to Chromium; a source that
+ is not one (Motion, or a share started in it) is capped, now from the first
+ frame, since the Mac capture's size is known before it delivers one.
+- The browser sharing to the app, modes cycled, renewals every ten seconds:
+ four and a half minutes, no freeze.
+- Considered and not done: making Chromium's H.264 level higher (it answers
+ 3.1 and room.js does not munge SDP, §5.1), and VP8 or VP9 for screens to
+ browsers (software VP8 managed 2 fps at 1080p, §5.4's measurements). The
+ measured cap holds; the ladder covers what it does not.
+
+What is left untraced, and what the new statistics are for: a receiver's own
+decoder going wrong (a hardware decoder in the watcher's browser, which a
+headless gate never uses), and the one ICE restart in ten between two apps that
+stops the screen for three or four seconds (§15.21).
+
+**Decision.** The sharer takes a report further, the receiver does its own
+share, and both say what they see.
+
+- **The sharer's ladder**, per reporting peer: keyframes as §15.21. Still
+ reported six seconds after the first, **that connection's screen is
+ rebuilt** — its transceiver stopped and a new one added once that has been
+ negotiated, at the same size and mode; a report never shrinks the screen or
+ makes a smaller encoder. Still reported ten seconds after that,
+ **everyone's screen is shared again** from the same capture: every screen
+ transceiver stopped, a new track on the same source (a clone of the captured
+ track in the browser), each connection given it as it settles; no picker.
+ Ten seconds after that, it gives up. A rebuild at most once a minute per
+ peer, a share again at most once every five minutes; a step that is not
+ allowed goes to the next, and none is taken unless at least three frames
+ left for that peer since the last step (a screen with nothing new to send,
+ or held back by the link, is not a decoder's fault). A report that clears
+ ends the ladder for that peer.
+- **The notice moves on in words**: "Samuel's view of your screen stalled —
+ recovering", then "Restarting your share for Samuel…", then either nothing
+ or "Couldn't restore Samuel's view — try Stop and Share again". Worst news
+ first when people are at different steps.
+- **The receiver**: any change in a screen's decoded count is movement. Four
+ seconds into a stall it hands the screen's picture to its view again, once
+ per stall (a new renderer in the apps, a fresh stream on the element in the
+ browser): a picture stopped on this side is this side's to mend. Neither
+ WebRTC build has a way for a receiver to ask for a keyframe directly; the
+ report is that request, and WebRTC's own PLIs go out as they always did. A
+ report is held for up to three minutes from the last frame, not one, so the
+ sharer's "Couldn't restore" is not taken away while it is still true.
+- **The iPhone's broadcast repeats its last frame once a second** while the
+ screen is still, as the Mac's capture has since §15.21.
+- **What is logged.** Every step, both sides, in the screen log (`os_log`
+ category `screen`; the browser's console), and every stall with what the
+ receiver's statistics say moved since the screen last did: packets, frames
+ received, decoded, dropped, keyframes, PLI and NACK, the decoder, and a
+ verdict (nothing arriving; arriving but not assembled; assembled but not
+ decoded). "Copy call diagnostics" carries the ladder's counts and stage per
+ peer, every arriving screen's statistics, and the last 200 lines of the
+ timeline, on the sharer and the receiver alike. The browser gains the same
+ button, at the foot of its Devices panel.
+
+**Compatibility.** Nothing on the wire changes. An older receiver reports as
+before and is answered with the new ladder; an older sharer answers a new
+receiver's longer report with keyframes as before.
+
+**Not logged, not stored.** As §15.19; the timeline stays on the device until
+someone copies it.
+
+### 15.23 §4.5, §5 — "Reconnecting…" only for a problem that lasts (2026-09-30)
+
+**Why.** Paul, 2026-09-30: "During that call I also saw 'Reconnecting' appear
+occasionally over the video, but only for a second … it didn't affect the call
+quality … I didn't notice any interruption in the call." The apps' core enters
+its `reconnecting` phase the moment the signalling stream ends, and the stream
+ends on purpose at its TTL (every five minutes, §4.5) and is resumed half a
+second later; the apps said "Reconnecting…" every time. The media never
+noticed. The browser said nothing at all, even for a real outage.
+
+**Decision.** The notice is said, on Mac, iPhone and web, only when:
+
+- the signalling stream has been down for 4 s or more — the whole call; or
+- a peer connection's ICE state has been `disconnected` or `failed` for 3 s or
+ more, or no inbound audio packets have arrived for 2 s or more from a peer
+ that is not muted — that peer's tile, "Reconnecting to Samuel…"; or the whole
+ call, when it is every peer (two or more).
+
+Once said it stays a second past the recovery, then goes. A routine renewal or
+resume is never said. The timings are named constants (`ReconnectWatch` in
+the apps, `RECONNECT_*` in room.js) with unit tests either side; the browser
+gate's `renewal` scenario and the native gate's `resume` and `renewals` watch
+for the words through several renewals and fail on any. The core's phase is
+unchanged: controls that need an admitted call still follow it.
diff --git a/internal/calls/static/call-window.js b/internal/calls/static/call-window.js
index a34337d..58c2440 100644
--- a/internal/calls/static/call-window.js
+++ b/internal/calls/static/call-window.js
@@ -370,6 +370,8 @@
if (s.terminal) return 'Call ended';
if (/full/i.test(status || '')) return 'The room is full';
if (!s.admitted) return /waiting/i.test(status || '') ? 'Waiting to be let in' : 'Connecting';
+ // §15.23: only ever a problem that has lasted (room.js's watch).
+ if (s.reconnect && s.reconnect.call) return 'Reconnecting';
return 'In the meeting';
}
const stateLine = $('call-state');
diff --git a/internal/calls/static/calls.css b/internal/calls/static/calls.css
index 6766953..4679721 100644
--- a/internal/calls/static/calls.css
+++ b/internal/calls/static/calls.css
@@ -457,6 +457,9 @@ body:has(> .calls-window) { background: var(--rst-bg); }
text-align: center;
}
.calls-window .calls-device-note { color: #fff; }
+/* "Copy call diagnostics" at the foot of the Devices popover (§15.22). */
+.calls-copy-diagnostics { margin-block-start: var(--rst-sp-3); align-self: flex-start; }
+.calls-window .calls-copy-diagnostics { color: #fff; }
/* The sharing notice (2026-09-28): a dark pill with its own Stop
sharing button, and the recovery line under it. */
diff --git a/internal/calls/static/room.js b/internal/calls/static/room.js
index 7fdc6da..54a16d4 100644
--- a/internal/calls/static/room.js
+++ b/internal/calls/static/room.js
@@ -71,6 +71,7 @@
const shareNotice = $('share-notice');
const shareStop = $('share-stop');
const shareRecoveryEl = $('share-recovery');
+ const copyDiagnostics = $('copy-diagnostics');
// §15.19: the tile marks, rendered by the server so they are the
// vendored badge and the app's own icons rather than a copy of either,
// and the polite region that says a change out loud. #status is no
@@ -277,10 +278,43 @@
// for a keyframe, at most once every SCREEN_KEYFRAME_EVERY, and rests
// SCREEN_KEYFRAME_REST after SCREEN_KEYFRAME_TRIES of them.
const SCREEN_STALL_AFTER = 4000;
- const SCREEN_STALL_GIVE_UP = 60000;
+ const SCREEN_STALL_GIVE_UP = 180000;
const SCREEN_KEYFRAME_EVERY = 4000;
const SCREEN_KEYFRAME_TRIES = 3;
const SCREEN_KEYFRAME_REST = 30000;
+ // §15.22 (2026-09-30), when keyframes do not answer. The receiver
+ // hands a stalled screen's picture to its element again once,
+ // SCREEN_REATTACH_AFTER into the stall. The sharer, still named
+ // SCREEN_REBUILD_AFTER after its first keyframe, rebuilds that
+ // connection's screen (a new transceiver: a new encoder here, a new
+ // receiver and decoder there); still named SCREEN_RESHARE_AFTER after
+ // that, shares again for everyone from the same capture; and
+ // SCREEN_GIVE_UP_AFTER after that, says it could not. A rebuild at most
+ // once every SCREEN_REBUILD_EVERY per person, a share again at most
+ // once every SCREEN_RESHARE_EVERY, and neither unless at least
+ // SCREEN_ESCALATE_SENT frames left for that person since the last step:
+ // a receiver cannot be failing to decode frames that were never sent.
+ const SCREEN_REATTACH_AFTER = 4000;
+ const SCREEN_REBUILD_AFTER = 6000;
+ const SCREEN_RESHARE_AFTER = 10000;
+ const SCREEN_GIVE_UP_AFTER = 10000;
+ const SCREEN_REBUILD_EVERY = 60000;
+ const SCREEN_RESHARE_EVERY = 300000;
+ const SCREEN_ESCALATE_SENT = 3;
+ // The screen's timeline, for "Copy call diagnostics": its last lines.
+ const SCREEN_LOG_LINES = 200;
+ // §15.23 (2026-09-30), "Reconnecting…" only for a problem that lasts:
+ // the stream down RECONNECT_STREAM_AFTER (the whole call); a connection
+ // disconnected or failed RECONNECT_CONNECTION_AFTER, or no audio packets
+ // from someone unmuted RECONNECT_AUDIO_AFTER (that person's tile, or the
+ // whole call when it is everyone). Said for RECONNECT_HOLD past the
+ // recovery, so it does not flicker. A stream renewal — ended on purpose
+ // and resumed in half a second — is never said at all.
+ const RECONNECT_STREAM_AFTER = 4000;
+ const RECONNECT_CONNECTION_AFTER = 3000;
+ const RECONNECT_AUDIO_AFTER = 2000;
+ const RECONNECT_HOLD = 1000;
+ const RECONNECT_EVERY = 200;
// §5.4, sending: "Optimise shared screen for". What a screen gives up
// first when the uplink or the encoder cannot carry all of it. The
// first is the default; sharePlan says what each one means.
@@ -345,6 +379,20 @@
const screenAsks = new Map();
let shareRecovery = '';
let screenStallTestUntil = 0;
+ // §15.22: each arriving screen's statistics at the last poll and when
+ // it last moved (share key -> screenReading); pictures handed back;
+ // when each peer's screen was last rebuilt, and everyone's last shared
+ // again; and the timeline itself.
+ const screenReadings = new Map();
+ const screenMoving = new Map();
+ let screenReattaches = 0;
+ const shareRebuiltAt = new Map();
+ let shareResharedAt = 0;
+ const shareCounts = { keyframes: 0, rebuilds: 0, reshares: 0, failures: 0 };
+ const screenLines = [];
+ // §15.23: when the stream went down (0 while it is up), and the watch.
+ let streamDownSince = 0;
+ const reconnect = reconnectRecord();
// §15.20: peer id -> what this tab is hearing from that peer, as
// pollLevels reads it (hearingRecord). selfHearing is the same record
// for our own microphone, read off the sender.
@@ -418,6 +466,7 @@
// trying since it opened, and that is the stretch STALL_WINDOW means.
let lastHelloAt = Date.now();
let reconnectTimer = 0;
+ let reconnectWatchTimer = 0;
let statsTimer = 0;
let fullTimer = 0;
let renewTimer = 0; // §5.3: the scheduled renewal, at `renewAt`
@@ -1248,6 +1297,7 @@
if (source !== es) return;
closeStream();
if (state.stopped) return;
+ if (!streamDownSince) streamDownSince = Date.now();
// Only a stream that actually lived clears the backoff. One that
// died moments after its hello has not proved anything, and
// resetting on the hello alone is how a tight reconnect loop
@@ -1385,6 +1435,7 @@
// then stayed open.
stalls = 0;
lastHelloAt = Date.now();
+ streamDownSince = 0;
const fresh = !state.peer;
state.peer = msg.peer || null;
state.boot = msg.boot || state.boot;
@@ -4303,25 +4354,110 @@
// judgeScreen is the receiver's verdict on one screen, from its decoded
// count this poll: stalled, or not. `seen` is its record, updated here.
// Pure over its arguments, and lifted by name in the node test.
+ //
+ // Any change in the count is movement, a smaller one included: a
+ // screen the sharer has rebuilt arrives on a new receiver whose count
+ // starts again, and up to 2026-09-30 only a larger count moved, so a
+ // rebuilt screen that was playing again stayed "stalled" here — and the
+ // sharer's notice with it — until the new count passed the old.
function judgeScreen(seen, decoded, now, frozen) {
- if (!seen.movedAt || (decoded > seen.decoded && !frozen)) {
+ if (!seen.movedAt || (decoded !== seen.decoded && !frozen)) {
seen.decoded = decoded;
seen.movedAt = now;
+ seen.reattached = false;
return false;
}
const quiet = now - seen.movedAt;
return quiet >= SCREEN_STALL_AFTER && quiet < SCREEN_STALL_GIVE_UP;
}
+ // reattachDue says, once per stall, when a stalled screen's picture
+ // should be handed to its element again (§15.22): SCREEN_REATTACH_AFTER
+ // into the stall. A picture stopped on this side — an element or a
+ // track Chromium has left muted — is this side's to mend. Pure over
+ // `seen`, and lifted by name in the node test.
+ function reattachDue(seen, now) {
+ const quiet = now - seen.movedAt;
+ if (seen.reattached || quiet < SCREEN_STALL_AFTER + SCREEN_REATTACH_AFTER || quiet >= SCREEN_STALL_GIVE_UP) {
+ return false;
+ }
+ seen.reattached = true;
+ return true;
+ }
+
+ // screenLog is one line of the screen's timeline: the console, and the
+ // last SCREEN_LOG_LINES kept for "Copy call diagnostics".
+ function screenLog(line) {
+ const d = new Date();
+ const pad = (n, w) => String(n).padStart(w || 2, '0');
+ const stamp = pad(d.getHours()) + ':' + pad(d.getMinutes()) + ':' + pad(d.getSeconds()) + '.' + pad(d.getMilliseconds(), 3);
+ screenLines.push(stamp + ' ' + line);
+ if (screenLines.length > SCREEN_LOG_LINES) screenLines.splice(0, screenLines.length - SCREEN_LOG_LINES);
+ console.info('calls: ' + line);
+ }
+
+ // screenReading is what one inbound screen report says, and
+ // readingSummary and readingMoved put it in words. Pure, and lifted by
+ // name in the node test: packets arriving and frames assembled but none
+ // decoded is a decoder; nothing arriving is the sender or the path.
+ function screenReading(r, trackMuted) {
+ return {
+ mid: r.mid || '',
+ packets: r.packetsReceived || 0,
+ lost: r.packetsLost || 0,
+ received: r.framesReceived || 0,
+ decoded: r.framesDecoded || 0,
+ dropped: r.framesDropped || 0,
+ keyframes: r.keyFramesDecoded || 0,
+ pli: r.pliCount || 0,
+ fir: r.firCount || 0,
+ nack: r.nackCount || 0,
+ freezes: r.freezeCount || 0,
+ width: r.frameWidth || 0,
+ height: r.frameHeight || 0,
+ decoder: r.decoderImplementation || '',
+ muted: !!trackMuted,
+ };
+ }
+
+ function readingSummary(r) {
+ return 'mid ' + (r.mid || '-') + ', ' + r.width + 'x' + r.height + ', decoder ' + (r.decoder || 'unknown') +
+ (r.muted ? ', track muted' : '') + '; packets ' + r.packets + ' (lost ' + r.lost + '), frames received ' +
+ r.received + ', decoded ' + r.decoded + ', dropped ' + r.dropped + ', keyframes ' + r.keyframes +
+ ', PLI ' + r.pli + ', FIR ' + r.fir + ', NACK ' + r.nack + ', freezes ' + r.freezes;
+ }
+
+ function readingMoved(r, b) {
+ if (!b || b.mid !== r.mid) return readingSummary(r) + ' — no earlier reading';
+ const d = (a, z) => (a >= z ? '+' + (a - z) : 'new ' + a);
+ let verdict = 'decoding';
+ if (r.packets === b.packets) verdict = 'no packets arriving (the sender or the path)';
+ else if (r.received === b.received) verdict = 'packets arriving, no frames assembled (loss, or waiting on a keyframe)';
+ else if (r.decoded === b.decoded) verdict = 'frames arriving, none decoded (the decoder)';
+ return 'packets ' + d(r.packets, b.packets) + ', frames received ' + d(r.received, b.received) + ', decoded ' +
+ d(r.decoded, b.decoded) + ', dropped ' + d(r.dropped, b.dropped) + ', keyframes ' + d(r.keyframes, b.keyframes) +
+ ', PLI ' + d(r.pli, b.pli) + ', NACK ' + d(r.nack, b.nack) + (r.muted ? ', track muted' : '') + ' — ' + verdict;
+ }
+
// watchScreens takes one poll's decoded counts, by share key, and
// reports the list when it changes.
- function watchScreens(decodedByKey) {
+ //
+ // `readings` is the same poll's screenReading per share key: kept for
+ // the diagnostics, and set against the reading from when the screen last
+ // moved, so a stall is logged with what is and is not still arriving.
+ function watchScreens(decodedByKey, readings) {
const now = Date.now();
const frozen = now < screenStallTestUntil;
const stalled = [];
for (const key of Array.from(screenSeen.keys())) {
if (!decodedByKey.has(key)) screenSeen.delete(key);
}
+ for (const key of Array.from(screenReadings.keys())) {
+ if (!decodedByKey.has(key)) {
+ screenReadings.delete(key);
+ screenMoving.delete(key);
+ }
+ }
for (const [key, decoded] of decodedByKey) {
const id = key.slice(0, -SHARE_SUFFIX.length);
const entry = pcs.get(id);
@@ -4331,57 +4467,394 @@
}
let seen = screenSeen.get(key);
if (!seen) {
- seen = { decoded: 0, movedAt: 0 };
+ seen = { decoded: 0, movedAt: 0, reattached: false };
screenSeen.set(key, seen);
}
- if (judgeScreen(seen, decoded, now, frozen)) stalled.push(id);
+ const reading = readings && readings.get(key);
+ if (reading) screenReadings.set(key, reading);
+ if (judgeScreen(seen, decoded, now, frozen)) {
+ stalled.push(id);
+ if (reattachDue(seen, now)) {
+ screenLog('screen from ' + id.slice(0, 6) + ' still stalled: handing its picture to the tile again; ' +
+ (reading ? readingMoved(reading, screenMoving.get(key)) : 'no reading'));
+ const t = tiles.get(key);
+ if (t && t.video) {
+ screenReattaches += 1;
+ replayTile(key, t, entry, true);
+ }
+ }
+ } else if (reading) {
+ screenMoving.set(key, reading);
+ }
}
stalled.sort();
if (stalled.join() === screenStalled.join()) return;
for (const id of stalled) {
- if (!screenStalled.includes(id)) console.info('calls: screen from ' + id.slice(0, 6) + ' stalled here, asking for a keyframe');
+ if (screenStalled.includes(id)) continue;
+ const key = shareKey(id);
+ const reading = screenReadings.get(key);
+ screenLog('screen from ' + id.slice(0, 6) + ' stalled here: reporting it; since it last moved ' +
+ (reading ? readingMoved(reading, screenMoving.get(key)) : 'no reading'));
+ }
+ for (const id of screenStalled) {
+ if (stalled.includes(id)) continue;
+ const reading = screenReadings.get(shareKey(id));
+ screenLog('screen from ' + id.slice(0, 6) + ' moving here again' + (reading ? ': ' + readingSummary(reading) : ''));
}
screenStalled = stalled;
reportMedia();
}
// answerScreenStalls is the sharer's half: every peer whose report
- // names us gets a keyframe from our screen's encoder, and the quiet
- // notice says whose view is being put right.
+ // names us gets a keyframe from our screen's encoder, and — when
+ // keyframes do not answer (§15.22) — that connection's screen rebuilt,
+ // then everyone's shared again from the same capture, and then words
+ // saying it could not be restored. The quiet notice says which.
function answerScreenStalls() {
const me = state.peer;
- const names_ = [];
const now = Date.now();
+ let reshare = false;
for (const [id, m] of remoteMedia) {
const asking = !!(share && me && m && m.screenStalled && m.screenStalled.includes(me));
if (!asking) {
+ const gone = screenAsks.get(id);
+ if (gone && gone.stage && share) screenLog(id.slice(0, 6) + '\u2019s view restored after ' + stageWords(gone.stage));
screenAsks.delete(id);
continue;
}
- names_.push(names.get(id) || 'Someone');
const entry = pcs.get(id);
let ask = screenAsks.get(id);
if (!ask) {
- ask = { at: 0, tries: 0, restUntil: 0 };
+ ask = { at: 0, tries: 0, restUntil: 0, stage: '', stageAt: 0 };
screenAsks.set(id, ask);
+ if (entry) entry.shareSentSince = 0;
+ }
+ if (entry && now >= ask.restUntil && now - ask.at >= SCREEN_KEYFRAME_EVERY) {
+ ask.at = now;
+ ask.tries += 1;
+ if (ask.tries >= SCREEN_KEYFRAME_TRIES) {
+ ask.tries = 0;
+ ask.restUntil = now + SCREEN_KEYFRAME_REST;
+ }
+ shareCounts.keyframes += 1;
+ screenLog('keyframe for ' + id.slice(0, 6) + ', whose view of the screen stalled');
+ shareKeyframe(entry).catch(() => {});
}
- if (!entry || now < ask.restUntil || now - ask.at < SCREEN_KEYFRAME_EVERY) continue;
- ask.at = now;
- ask.tries += 1;
- if (ask.tries >= SCREEN_KEYFRAME_TRIES) {
- ask.tries = 0;
- ask.restUntil = now + SCREEN_KEYFRAME_REST;
+ // Frames left for this person since the last step, or no screen
+ // negotiated with them at all (a rebuild that never came back).
+ const negotiated = !!(entry && entry.shareTx && entry.shareSend);
+ const owed = !negotiated || (entry.shareSentSince || 0) >= SCREEN_ESCALATE_SENT;
+ const rec = { stage: ask.stage, at: ask.stageAt, lastRebuild: shareRebuiltAt.get(id) || 0 };
+ const before = rec.stage;
+ const reshareFree = reshare || !shareResharedAt || now - shareResharedAt >= SCREEN_RESHARE_EVERY;
+ const action = escalateShare(rec, now, owed, reshareFree);
+ ask.stage = rec.stage;
+ ask.stageAt = rec.at;
+ if (rec.stage !== before && entry) entry.shareSentSince = 0;
+ if (action === 'rebuild') {
+ shareRebuiltAt.set(id, now);
+ shareCounts.rebuilds += 1;
+ screenLog(id.slice(0, 6) + ' still reports a stall after keyframes: rebuilding its screen (new transceiver, same size and mode)');
+ if (entry) rebuildShareFor(entry);
+ } else if (action === 'reshare') {
+ reshare = true;
+ screenLog(id.slice(0, 6) + ' still reports a stall' + (before === 'rebuilt' ? ' after the rebuild' : ', rebuilt under a minute ago') + ': sharing again');
+ } else if (action === 'failed') {
+ shareCounts.failures += 1;
+ screenLog(id.slice(0, 6) + ' still reports a stall and there is nothing left to try: giving up');
}
- console.info('calls: keyframe for ' + id.slice(0, 6) + ', whose view of the screen stalled');
- shareKeyframe(entry).catch(() => {});
}
- const text = recoveringWords(names_);
+ if (reshare) {
+ shareResharedAt = now;
+ shareCounts.reshares += 1;
+ // One share again answers every report in flight.
+ for (const ask of screenAsks.values()) {
+ if (ask.stage === 'keyframes' || ask.stage === 'rebuilt') {
+ ask.stage = 'reshared';
+ ask.stageAt = now;
+ }
+ }
+ for (const entry of pcs.values()) entry.shareSentSince = 0;
+ reshareAll();
+ }
+ const recovering = [];
+ const restarting = [];
+ const failed = [];
+ for (const [id, ask] of screenAsks) {
+ const who = names.get(id) || 'Someone';
+ if (ask.stage === 'failed') failed.push(who);
+ else if (ask.stage === 'rebuilt' || ask.stage === 'reshared') restarting.push(who);
+ else recovering.push(who);
+ }
+ const text = noticeWords(recovering.sort(), restarting.sort(), failed.sort());
if (text !== shareRecovery) {
shareRecovery = text;
renderShare();
}
}
+ // callDiagnostics is "Copy call diagnostics" for the browser: the
+ // connections, and the screen — how ours is leaving and what the ladder
+ // has done, every screen arriving here as its statistics have it, and
+ // the timeline. Names, states and counts only: no media, no addresses,
+ // no credentials.
+ function callDiagnostics() {
+ const lines = ['Calls web diagnostics, ' + new Date().toISOString()];
+ lines.push('Browser: ' + (navigator.userAgent || 'unknown'));
+ lines.push('');
+ lines.push('## Call');
+ lines.push('admitted ' + state.admitted + ', people here ' + (pcs.size + 1) + ', stopped ' + state.stopped);
+ for (const [id, entry] of pcs) {
+ lines.push((names.get(id) || 'Someone') + ' (' + id.slice(0, 6) + '): connection ' + entry.pc.connectionState +
+ ', signalling ' + entry.pc.signalingState + ', ICE ' + entry.pc.iceConnectionState);
+ }
+ lines.push('');
+ lines.push('## Screen');
+ if (share) {
+ lines.push('Sharing (' + shareMode() + ')');
+ for (const [id, entry] of pcs) {
+ const r = entry.shareSend;
+ const who = names.get(id) || 'Someone';
+ if (!r) {
+ lines.push('to ' + who + ': ' + (entry.shareTx ? 'negotiating' : 'no screen transceiver'));
+ continue;
+ }
+ lines.push('to ' + who + ': ' + r.width + 'x' + r.height + ' at ' + Math.round(r.fps) + ' fps, encoder ' +
+ (r.encoder || 'unknown') + ', encoded ' + r.framesEncoded + ', sent ' + r.framesSent + ', keyframes ' +
+ r.keyFramesEncoded + ', limitation ' + (r.limitation || 'none'));
+ }
+ for (const [id, ask] of screenAsks) {
+ if (ask.stage) lines.push('recovering ' + (names.get(id) || 'Someone') + '\u2019s view: ' + ask.stage);
+ }
+ } else {
+ lines.push('Not sharing');
+ }
+ lines.push('recoveries: ' + shareCounts.keyframes + ' keyframes, ' + shareCounts.rebuilds + ' rebuilds, ' +
+ shareCounts.reshares + ' shares again, ' + shareCounts.failures + ' given up');
+ for (const [key, reading] of screenReadings) {
+ const id = key.slice(0, -SHARE_SUFFIX.length);
+ lines.push('from ' + (names.get(id) || 'Someone') + ': ' + readingSummary(reading));
+ }
+ if (screenStalled.length) lines.push('stalled here: ' + screenStalled.map((id) => names.get(id) || 'Someone').join(', '));
+ if (screenReattaches) lines.push('pictures handed to their tiles again here: ' + screenReattaches);
+ lines.push('');
+ lines.push('## Screen log (last ' + SCREEN_LOG_LINES + ' lines)');
+ if (screenLines.length) lines.push(...screenLines);
+ else lines.push('(nothing logged)');
+ return lines.join('\n');
+ }
+
+ // ---------------------------------------------------------------
+ // "Reconnecting…" (§15.23, 2026-09-30)
+ // ---------------------------------------------------------------
+ //
+ // Paul: "I saw 'Reconnecting' appear occasionally over the video, but
+ // only for a second … I didn't notice any interruption in the call."
+ // The Mac app said so the moment its stream ended, and a stream ends on
+ // purpose every few minutes and is resumed half a second later. Here,
+ // as there, the words wait for a problem that lasts.
+
+ function reconnectRecord() {
+ return { streamSince: 0, connection: new Map(), audio: new Map(), call: false, callClear: 0,
+ peers: new Set(), peerClear: new Map() };
+ }
+
+ // reconnectStep is one look: `streamDown`, and `people` (peer id ->
+ // {connectionBad, audioMissing}) for everyone else. It moves `rec` on:
+ // rec.call is the whole call's words, rec.peers the tiles that say
+ // "Reconnecting to …". The native app's ReconnectWatch is its twin.
+ // Pure, and lifted by name in the node test.
+ function reconnectStep(rec, now, streamDown, people) {
+ rec.streamSince = streamDown ? rec.streamSince || now : 0;
+ for (const id of Array.from(rec.connection.keys())) if (!people.has(id)) rec.connection.delete(id);
+ for (const id of Array.from(rec.audio.keys())) if (!people.has(id)) rec.audio.delete(id);
+ const troubled = new Set();
+ for (const [id, p] of people) {
+ if (p.connectionBad) {
+ if (!rec.connection.has(id)) rec.connection.set(id, now);
+ } else {
+ rec.connection.delete(id);
+ }
+ if (p.audioMissing) {
+ if (!rec.audio.has(id)) rec.audio.set(id, now);
+ } else {
+ rec.audio.delete(id);
+ }
+ const connection = rec.connection.has(id) && now - rec.connection.get(id) >= RECONNECT_CONNECTION_AFTER;
+ const audio = rec.audio.has(id) && now - rec.audio.get(id) >= RECONNECT_AUDIO_AFTER;
+ if (connection || audio) troubled.add(id);
+ }
+ const stream = !!rec.streamSince && now - rec.streamSince >= RECONNECT_STREAM_AFTER;
+ const everyone = troubled.size >= 2 && troubled.size === people.size;
+ const wantCall = stream || everyone;
+ if (wantCall) {
+ rec.call = true;
+ rec.callClear = 0;
+ } else if (rec.call) {
+ if (!rec.callClear) rec.callClear = now;
+ if (now - rec.callClear >= RECONNECT_HOLD) {
+ rec.call = false;
+ rec.callClear = 0;
+ }
+ }
+ const shown = new Set(Array.from(rec.peers).filter((id) => people.has(id)));
+ if (!wantCall) {
+ for (const id of troubled) {
+ shown.add(id);
+ rec.peerClear.delete(id);
+ }
+ }
+ for (const id of Array.from(shown)) {
+ if (!wantCall && troubled.has(id)) continue;
+ if (wantCall) {
+ shown.delete(id);
+ rec.peerClear.delete(id);
+ continue;
+ }
+ if (!rec.peerClear.has(id)) rec.peerClear.set(id, now);
+ if (now - rec.peerClear.get(id) >= RECONNECT_HOLD) {
+ shown.delete(id);
+ rec.peerClear.delete(id);
+ }
+ }
+ for (const id of Array.from(rec.peerClear.keys())) if (!shown.has(id)) rec.peerClear.delete(id);
+ rec.peers = shown;
+ }
+
+ // watchReconnect feeds the watch what this tab can see, five times a
+ // second, and redraws what changed.
+ function watchReconnect() {
+ if (state.stopped) return;
+ const now = Date.now();
+ const people = new Map();
+ for (const [id, entry] of pcs) {
+ const ice = entry.pc.iceConnectionState;
+ const rec = hearing.get(id);
+ const media = remoteMedia.get(id);
+ people.set(id, {
+ connectionBad: ice === 'disconnected' || ice === 'failed',
+ // Two level polls without a packet: stopped, as far as a poll can tell.
+ audioMissing: !!(rec && rec.packetAt && media && !media.muted && now - rec.packetAt > 500),
+ });
+ }
+ const wasCall = reconnect.call;
+ const wasPeers = Array.from(reconnect.peers).sort().join();
+ reconnectStep(reconnect, now, !!streamDownSince && !!state.peer, people);
+ if (reconnect.call !== wasCall) {
+ console.info('calls: reconnecting ' + (reconnect.call ? 'shown (a problem that lasted)' : 'cleared'));
+ renderStatus();
+ }
+ if (Array.from(reconnect.peers).sort().join() !== wasPeers) {
+ for (const id of pcs.keys()) drawHearing(id);
+ }
+ }
+
+ // noteShareSent counts the frames that left on this connection's
+ // screen since its escalation last moved on, across a new transceiver's
+ // counters starting again from nothing.
+ function noteShareSent(entry) {
+ if (!entry.shareSend) return;
+ const sent = entry.shareSend.framesSent || 0;
+ const last = entry.shareSentLast || 0;
+ entry.shareSentSince = (entry.shareSentSince || 0) + (sent >= last ? sent - last : sent);
+ entry.shareSentLast = sent;
+ }
+
+ // escalateShare is §15.22's ladder for one reporting peer: '' (nothing
+ // now), 'rebuild', 'reshare' or 'failed', moving `rec` ({stage, at,
+ // lastRebuild}) on. `owed` is whether frames have left for that person
+ // since the last step; `reshareFree` whether a share again may happen
+ // now. The native app's ScreenWatchdog is its twin. Pure, and lifted by
+ // name in the node test.
+ function escalateShare(rec, now, owed, reshareFree) {
+ const move = (stage) => {
+ rec.stage = stage;
+ rec.at = now;
+ };
+ if (!rec.stage) {
+ move('keyframes');
+ return '';
+ }
+ if (rec.stage === 'keyframes') {
+ if (now - rec.at < SCREEN_REBUILD_AFTER || !owed) return '';
+ if (!rec.lastRebuild || now - rec.lastRebuild >= SCREEN_REBUILD_EVERY) {
+ move('rebuilt');
+ rec.lastRebuild = now;
+ return 'rebuild';
+ }
+ } else if (rec.stage === 'rebuilt') {
+ if (now - rec.at < SCREEN_RESHARE_AFTER || !owed) return '';
+ } else if (rec.stage === 'reshared') {
+ if (now - rec.at < SCREEN_GIVE_UP_AFTER || !owed) return '';
+ move('failed');
+ return 'failed';
+ } else {
+ return '';
+ }
+ if (reshareFree) {
+ move('reshared');
+ return 'reshare';
+ }
+ move('failed');
+ return 'failed';
+ }
+
+ function stageWords(stage) {
+ return { keyframes: 'keyframes', rebuilt: 'the rebuild', reshared: 'sharing again', failed: 'giving up' }[stage] || stage;
+ }
+
+ // rebuildShareFor is a stop and share again for one person (§15.22):
+ // this connection's screen transceiver stopped, and a new one added by
+ // the connection's own signalingstatechange once that has been
+ // negotiated — a new encoder here, a new receiver and decoder there, at
+ // the same size and mode.
+ function rebuildShareFor(entry) {
+ if (!share || !entry.shareTx) return;
+ removeShareFrom(entry);
+ sayShareAgain();
+ }
+
+ // reshareAll is Stop and Share for everyone without the picker: every
+ // connection's screen transceiver stopped, and a clone of the captured
+ // track — the same capture, a new track — added to each as it settles.
+ function reshareAll() {
+ if (!share) return;
+ const held = share;
+ let track;
+ try {
+ track = held.track.clone();
+ } catch {
+ return;
+ }
+ try {
+ track.contentHint = held.track.contentHint;
+ } catch {
+ // A browser without contentHint.
+ }
+ // The browser's own Stop sharing bar ends every clone.
+ track.addEventListener('ended', () => {
+ if (share && share.track === track) stopSharing();
+ });
+ for (const entry of pcs.values()) removeShareFrom(entry);
+ share = { stream: new MediaStream([track]), track };
+ // The old one after: stopping it alone does not fire its own
+ // `ended`, and the clone keeps the capture running.
+ try {
+ held.track.stop();
+ } catch {
+ // Already stopped.
+ }
+ sayShareAgain();
+ }
+
+ // sayShareAgain repeats `share {on:true}` once a rebuilt screen has had
+ // time to arrive: its new track comes without a label of its own.
+ function sayShareAgain() {
+ setTimeout(() => {
+ if (share) broadcast('share', { on: true });
+ }, 2000);
+ }
+
// recoveringWords is the sharer's notice. Pure, and lifted by name in
// the node test.
function recoveringWords(people) {
@@ -4393,6 +4866,30 @@
return people.length + ' people\u2019s views of your screen stalled \u2014 recovering';
}
+ // noticeWords is the notice as §15.22's ladder moves on: recovering,
+ // then "Restarting your share for Samuel…", then either nothing or
+ // "Couldn’t restore Samuel’s view — try Stop and Share again". Worst
+ // news first. The native app's ScreenWords.notice, word for word. Pure,
+ // and lifted by name in the node test (with recoveringWords).
+ function noticeWords(recovering, restarting, failed) {
+ const parts = [];
+ if (failed.length) {
+ let whose = failed.length + ' people\u2019s views';
+ if (failed.length === 1) whose = failed[0] + '\u2019s view';
+ else if (failed.length === 2) whose = failed[0] + '\u2019s and ' + failed[1] + '\u2019s views';
+ parts.push('Couldn\u2019t restore ' + whose + ' \u2014 try Stop and Share again');
+ }
+ if (restarting.length) {
+ let who = restarting.length + ' people';
+ if (restarting.length === 1) who = restarting[0];
+ else if (restarting.length === 2) who = restarting[0] + ' and ' + restarting[1];
+ parts.push('Restarting your share for ' + who + '\u2026');
+ }
+ const stalled = recoveringWords(recovering);
+ if (stalled) parts.push(stalled);
+ return parts.join('. ');
+ }
+
// shareKeyframe switches this connection's screen encoding off and on
// again, which WebRTC answers by starting it on a keyframe: there is
// no API that asks for one directly.
@@ -4780,7 +5277,9 @@
h.box.hidden = !kind;
}
if (self) return;
- const notice = remoteNotice(name, media, state.peer, localMedia().muted);
+ const notice = reconnect.peers.has(key)
+ ? 'Reconnecting to ' + name + '\u2026'
+ : remoteNotice(name, media, state.peer, localMedia().muted);
if (h.notice.textContent !== notice) {
h.notice.textContent = notice;
h.notice.hidden = !notice;
@@ -5103,6 +5602,7 @@
clearTimeout(reconnectTimer);
clearTimeout(fullTimer);
clearInterval(statsTimer);
+ clearInterval(reconnectWatchTimer);
clearInterval(levelTimer);
stopIceTimers();
closeStream();
@@ -5197,6 +5697,10 @@
: 'Connecting to the call.');
return;
}
+ if (reconnect.call) {
+ say('Reconnecting…');
+ return;
+ }
const others = pcs.size;
if (!others) {
say('You are the only one here.', true);
@@ -5694,6 +6198,21 @@
});
}
+ // "Copy call diagnostics" (§15.22), and a sentence in the device
+ // panel's own live region either way.
+ if (copyDiagnostics) {
+ copyDiagnostics.addEventListener('click', () => {
+ const text = callDiagnostics();
+ const done = () => note('Call diagnostics copied.');
+ const failed = () => note('The diagnostics could not be copied.');
+ try {
+ navigator.clipboard.writeText(text).then(done, failed);
+ } catch {
+ failed();
+ }
+ });
+ }
+
leaveButton.addEventListener('click', () => {
leave();
go('/m/' + encodeURIComponent(sqid) + '/join');
@@ -5716,6 +6235,7 @@
clearTimeout(reconnectTimer);
clearTimeout(fullTimer);
clearInterval(statsTimer);
+ clearInterval(reconnectWatchTimer);
clearInterval(levelTimer);
stopIceTimers();
if (state.peer) postSignal({ peer: state.peer, type: 'leave' }, { keepalive: true });
@@ -5921,6 +6441,8 @@
targetBitrate: r.targetBitrate || 0,
encoder: r.encoderImplementation || '',
framesEncoded: r.framesEncoded || 0,
+ framesSent: r.framesSent || 0,
+ keyFramesEncoded: r.keyFramesEncoded || 0,
};
}
@@ -6165,8 +6687,10 @@
// callback was dropped is put back on the books (Codex S4-6's gate
// found one, and the counter is what every frames assertion reads).
rearmFrames();
- // §15.21: every screen's decoded count, for the receiver's watch.
+ // §15.21: every screen's decoded count, for the receiver's watch, and
+ // (§15.22) the rest of what its statistics say.
const screens = new Map();
+ const screenReports = new Map();
for (const [id, entry] of pcs) {
try {
const stats = await entry.pc.getStats();
@@ -6188,7 +6712,11 @@
const key = frameKey(id, entry, report.mid || '');
if (!key) return;
decoded.set(key, report.framesDecoded);
- if (key !== id) screens.set(key, report.framesDecoded);
+ if (key !== id) {
+ screens.set(key, report.framesDecoded);
+ const t = tiles.get(key);
+ screenReports.set(key, screenReading(report, !!(t && t.track && t.track.muted)));
+ }
if (!HAS_RVFC) frames.set(key, report.framesDecoded);
});
for (const [key, count] of decoded) {
@@ -6199,6 +6727,7 @@
else if (verdict === 'reset') requestReset(id);
}
entry.shareSend = shareSendReport(outbound, (entry.shareTx && entry.shareTx.mid) || '');
+ noteShareSent(entry);
const tx = localCameraTx(entry);
const verdict = judgeCamera(entry, tx, cameraReport(outbound, (tx && tx.mid) || ''));
if (verdict === 'repair') await repairCamera(tx);
@@ -6207,7 +6736,7 @@
// getStats on a closing connection: nothing to report.
}
}
- watchScreens(screens);
+ watchScreens(screens, screenReports);
answerScreenStalls();
}
@@ -6341,6 +6870,10 @@
// §15.21, for the gate: this tab's receiver watch treats every
// screen as frozen for `ms`, so its stall report and the sharer's
// keyframe can be driven end to end.
+ // §15.22: "Copy call diagnostics" as it would be copied.
+ diagnostics() {
+ return callDiagnostics();
+ },
stallScreenForTest(ms) {
screenStallTestUntil = Date.now() + Math.max(0, Math.min(Number(ms) || 0, 60000));
},
@@ -6371,6 +6904,10 @@
// sharer's notice while someone's view of ours is being put right.
screenStalled: screenStalled.slice(),
shareRecovery,
+ // §15.22: what the ladder and the receiver's own mending have done.
+ shareRecoveries: Object.assign({ reattaches: screenReattaches }, shareCounts),
+ // §15.23: whether "Reconnecting…" is said, for the call and by tile.
+ reconnect: { call: reconnect.call, peers: Array.from(reconnect.peers), streamDown: !!streamDownSince },
// Targets a 404 has marked stale: nothing is sent to them and
// no reset is asked for until a roster settles it.
stale: Array.from(stale),
@@ -6469,6 +7006,8 @@
// camera that stopped encoding (Codex S4-5), so the walk runs
// either way and does whichever halves apply.
statsTimer = setInterval(pollFrames, 1000);
+ // §15.23: no stats, only what the tab already knows.
+ reconnectWatchTimer = setInterval(watchReconnect, RECONNECT_EVERY);
// §15.20's meters, five times a second and cheaper per turn: one
// receiver's and one sender's own stats, not the connection's walk.
levelTimer = setInterval(() => {
diff --git a/internal/calls/templates/room.html b/internal/calls/templates/room.html
index ea18507..3bf2521 100644
--- a/internal/calls/templates/room.html
+++ b/internal/calls/templates/room.html
@@ -172,6 +172,12 @@
render, so a sentence about a camera that would not open has
to live somewhere that is not about to be overwritten. */}}
<p id="device-note" class="calls-device-note" role="status" hidden></p>
+ {{/* What the Mac panel's "Copy call diagnostics" copies, for the
+ browser: the connections, and the screen's recovery timeline and
+ decode statistics on both sides of a share (spec §15.22). Names,
+ states and counts only; room.js says in #device-note whether it
+ was copied. */}}
+ <button type="button" id="copy-diagnostics" rst-btn="ghost" class="calls-copy-diagnostics">Copy call diagnostics</button>
</div>
{{/* Ending the meeting for everyone is destructive, so it is a link to
its own confirm page (AGENTS.md), opened in a new tab so that
diff --git a/internal/callstest/native_browser_test.go b/internal/callstest/native_browser_test.go
index e4cadaf..efb643f 100644
--- a/internal/callstest/native_browser_test.go
+++ b/internal/callstest/native_browser_test.go
@@ -11,6 +11,8 @@ import (
"os"
"os/exec"
"path/filepath"
+ "sort"
+ "strconv"
"strings"
"sync"
"testing"
@@ -221,6 +223,8 @@ type nativeSample struct {
Stats map[string]uint64 `json:"stats"`
ScreenCaptions []string `json:"screenCaptions"`
SharingScreen *bool `json:"sharingScreen"`
+ ScreenRecovery string `json:"screenRecovery"`
+ Diagnostics string `json:"diagnostics"`
}
// nativeApp is one CallsInterop process and the last observation it
@@ -231,6 +235,7 @@ type nativeApp struct {
stdin io.WriteCloser
mu sync.Mutex
last nativeSample
+ diag string
done chan struct{}
}
@@ -256,7 +261,16 @@ func startNative(ctx context.Context, t *testing.T, binary string, rig *browserR
scanner := bufio.NewScanner(out)
for scanner.Scan() {
var s nativeSample
- if json.Unmarshal(scanner.Bytes(), &s) != nil || s.Stats == nil {
+ if json.Unmarshal(scanner.Bytes(), &s) != nil {
+ continue
+ }
+ if s.Diagnostics != "" {
+ app.mu.Lock()
+ app.diag = s.Diagnostics
+ app.mu.Unlock()
+ continue
+ }
+ if s.Stats == nil {
continue
}
app.mu.Lock()
@@ -278,6 +292,25 @@ func (n *nativeApp) sample() nativeSample {
func (n *nativeApp) stat(name string) uint64 { return n.sample().Stats[name] }
+// diagnostics asks for "Copy call diagnostics" and returns what came
+// back, or "" if nothing did within five seconds.
+func (n *nativeApp) diagnostics(t *testing.T) string {
+ t.Helper()
+ n.mu.Lock()
+ n.diag = ""
+ n.mu.Unlock()
+ n.command(t, "diagnostics")
+ for until := time.Now().Add(5 * time.Second); time.Now().Before(until); time.Sleep(100 * time.Millisecond) {
+ n.mu.Lock()
+ d := n.diag
+ n.mu.Unlock()
+ if d != "" {
+ return d
+ }
+ }
+ return ""
+}
+
func (n *nativeApp) command(t *testing.T, line string) {
t.Helper()
if _, err := io.WriteString(n.stdin, line+"\n"); err != nil {
@@ -373,24 +406,31 @@ func TestNativeMacScreenRecovery(t *testing.T) {
if _, err := os.Stat(binary); err != nil {
t.Fatal("build native interop executable with make native-mac first")
}
- scenarios := []string{"renewals", "resume", "capture", "encoder", "loss"}
+ scenarios := []string{"renewals", "resume", "capture", "encoder", "loss", "wedge", "wedge-sticky", "web-wedge", "web-wedge-sticky"}
if only := os.Getenv("SCREEN_SCENARIOS"); only != "" {
scenarios = strings.Split(only, ",")
}
for _, scenario := range scenarios {
t.Run(scenario, func(t *testing.T) {
rig := newBrowserRig(t)
- if scenario == "renewals" {
+ soaking := scenario == "soak" || scenario == "web-soak"
+ if scenario == "renewals" || soaking {
if err := rig.ctl.SetRelay(browserRelayContract()); err != nil {
t.Fatal(err)
}
rig.ctl.SetIceLife(20 * time.Second)
}
- if scenario == "renewals" || scenario == "resume" {
+ if scenario == "renewals" || scenario == "resume" || soaking {
rig.ctl.SetStreamTimings(time.Second, 10*time.Second)
}
room, _ := rig.createMeeting(t)
- ctx, cancel := context.WithTimeout(t.Context(), 240*time.Second)
+ limit := 240 * time.Second
+ if soaking {
+ // SCREEN_SOAK_SECONDS of soak, and the scenario around it.
+ seconds, _ := strconv.Atoi(envOr("SCREEN_SOAK_SECONDS", "60"))
+ limit = time.Duration(seconds)*time.Second + 180*time.Second
+ }
+ ctx, cancel := context.WithTimeout(t.Context(), limit)
defer cancel()
cmd := exec.CommandContext(ctx, "node", filepath.Join(root, "test/browser/native-screen.mjs"), rig.origin, room, binary, scenario)
cookies, _ := json.Marshal([]map[string]string{{"name": rig.cookieName, "value": rig.cookieB}})
@@ -454,7 +494,7 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
if v, err := time.ParseDuration(os.Getenv("SCREEN_PAIR_SOAK")); err == nil {
soak = v
}
- scenarios := []string{"renewals", "capture", "encoder", "loss"}
+ scenarios := []string{"renewals", "capture", "encoder", "loss", "wedge", "wedge-sticky", "reshare"}
if only := os.Getenv("SCREEN_SCENARIOS"); only != "" {
scenarios = strings.Split(only, ",")
}
@@ -466,9 +506,15 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
t.Skip("the sharing app predates the stall commands")
case scenario == "loss" && (!newA || !newB):
t.Skip("a stall report needs the new receiver and the new sharer")
+ case strings.HasPrefix(scenario, "wedge") || scenario == "reshare":
+ if !newA || !newB {
+ t.Skip("the escalation needs the new receiver and the new sharer")
+ }
+ // Decoders the receiver can wedge (spec §15.22).
+ t.Setenv("CALLS_DECODER_WEDGE", "1")
}
rig := newBrowserRig(t)
- if scenario == "renewals" {
+ if scenario == "renewals" || scenario == "soak" {
if err := rig.ctl.SetRelay(browserRelayContract()); err != nil {
t.Fatal(err)
}
@@ -501,6 +547,9 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
a.command(t, "screen-mode "+envOr("SCREEN_MODE", "balanced"))
if os.Getenv("SCREEN_PAIR_DISPLAY") == "1" && newA {
a.command(t, "screen-display 6")
+ } else if os.Getenv("SCREEN_SOURCE") == "motion" {
+ // The 1920-wide scrolling page (CALLS_SYNTHETIC_SCREEN_HEIGHT).
+ a.command(t, "screen-motion")
} else {
a.command(t, "screen-on")
}
@@ -510,11 +559,11 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
// The watch: B's screen frames, once a second.
freezes, longest := 0, time.Duration(0)
watch := func(d time.Duration, label string) {
- last, moved, frozen := b.stat("screenFramesDecoded"), time.Now(), false
+ last, moved, frozen := b.stat("screenFramesDecodedTotal"), time.Now(), false
camera := b.stat("framesDecoded")
for until := time.Now().Add(d); time.Now().Before(until); {
time.Sleep(time.Second)
- now := b.stat("screenFramesDecoded")
+ now := b.stat("screenFramesDecodedTotal")
if now > last {
if frozen {
gap := time.Since(moved)
@@ -529,9 +578,10 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
} else if !frozen && time.Since(moved) > 3*time.Second {
frozen = true
s := a.sample().Stats
- t.Logf("FROZEN %s: no screen frame at b for %v (a: captured %d encoded %d sent %d, watchdog k%d e%d r%d a%d)", label,
+ t.Logf("FROZEN %s: no screen frame at b for %v (a: captured %d encoded %d sent %d, watchdog k%d e%d r%d a%d rebuilds %d shares again %d)", label,
time.Since(moved).Round(time.Millisecond), s["screenCaptureDelivered"], s["screenFramesEncoded"], s["screenFramesSent"],
- s["screenWatchdogKeyframes"], s["screenWatchdogEncoderRecoveries"], s["screenWatchdogCaptureRestarts"], s["screenWatchdogReAdds"])
+ s["screenWatchdogKeyframes"], s["screenWatchdogEncoderRecoveries"], s["screenWatchdogCaptureRestarts"], s["screenWatchdogReAdds"],
+ s["screenWatchdogRebuilds"], s["screenWatchdogReshares"])
}
last = now
}
@@ -551,6 +601,61 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
t.Fatalf("%s: a=%+v b=%+v", label, a.sample(), b.sample())
}
switch scenario {
+ case "soak":
+ // §15.22's search for the cause, between two apps: renewals
+ // and resumes as `renewals`, with A's mode cycled and its
+ // uplink capped and lifted every 45 s, for SCREEN_PAIR_SOAK.
+ modes := []string{"balanced", "motion", "detail"}
+ uplinks := []string{"0", "1500000", "600000", "0", "3000000", "900000"}
+ done := make(chan struct{})
+ notices := map[string]bool{}
+ var mu sync.Mutex
+ go func() {
+ step := 0
+ tick := time.NewTicker(45 * time.Second)
+ defer tick.Stop()
+ for {
+ select {
+ case <-done:
+ return
+ case <-tick.C:
+ step++
+ a.command(t, "screen-mode "+modes[step%len(modes)])
+ a.command(t, "uplink "+uplinks[step%len(uplinks)])
+ t.Logf("soak step %d: mode %s, uplink %s", step, modes[step%len(modes)], uplinks[step%len(uplinks)])
+ }
+ }
+ }()
+ go func() {
+ for {
+ select {
+ case <-done:
+ return
+ case <-time.After(500 * time.Millisecond):
+ if n := a.sample().ScreenRecovery; n != "" {
+ mu.Lock()
+ notices[n] = true
+ mu.Unlock()
+ }
+ }
+ }
+ }()
+ watch(soak, "soak")
+ close(done)
+ a.command(t, "uplink 0")
+ s := a.sample().Stats
+ t.Logf("SOAK a: ICE restarts %d (b %d), stall reports %d, keyframes %d, fresh encoders %d, re-adds %d, rebuilds %d, shares again %d, given up %d, capture restarts %d; b's pictures attached again %d",
+ s["iceRestarts"], b.stat("iceRestarts"), s["screenStallReports"], s["screenWatchdogKeyframes"], s["screenWatchdogEncoderRecoveries"],
+ s["screenWatchdogReAdds"], s["screenWatchdogRebuilds"], s["screenWatchdogReshares"], s["screenWatchdogFailures"],
+ s["screenWatchdogCaptureRestarts"], b.stat("screenReattaches"))
+ mu.Lock()
+ t.Logf("SOAK notices at a: %q; %d freezes, longest %v", keys(notices), freezes, longest)
+ mu.Unlock()
+ t.Logf("a's diagnostics:\n%s", screenSections(a.diagnostics(t)))
+ t.Logf("b's diagnostics:\n%s", screenSections(b.diagnostics(t)))
+ if s["screenWatchdogFailures"] > 0 {
+ t.Error("a gave up on b's view")
+ }
case "renewals":
watch(soak, "renewals")
t.Logf("ICE restarts: a started %d, b started %d", a.stat("iceRestarts"), b.stat("iceRestarts"))
@@ -592,12 +697,154 @@ func TestNativeMacScreenPairRecovery(t *testing.T) {
}, a, b)
waitNative(t, "b clears its report once frames move", func() bool { return b.stat("screenStalledHere") == 0 }, a, b)
recovered("b decodes after the keyframe")
+ case "wedge":
+ // B's screen decoder stops putting out pictures while packets
+ // keep arriving (spec §15.22): keyframes cannot mend it and a
+ // new transceiver can. Without a manual share again, B has to
+ // be decoding again within 30 s, and A's notice has to have
+ // said so honestly on the way.
+ watch(5*time.Second, "before")
+ keyframes := a.stat("screenWatchdogKeyframes")
+ wedged := time.Now()
+ b.command(t, "screen-decoder-wedge")
+ waitNative(t, "b reports a's screen stalled", func() bool { return b.stat("screenStalledHere") == 1 }, a, b)
+ notices := map[string]bool{}
+ base := b.stat("screenFramesDecodedTotal")
+ back := false
+ for until := wedged.Add(30 * time.Second); time.Now().Before(until); time.Sleep(250 * time.Millisecond) {
+ if n := a.sample().ScreenRecovery; n != "" {
+ notices[n] = true
+ }
+ if b.stat("screenFramesDecodedTotal") > base+15 {
+ back = true
+ break
+ }
+ }
+ took := time.Since(wedged)
+ t.Logf("b decoding again after %v: rebuilds %d, shares again %d, keyframes %d, b's pictures attached again %d",
+ took.Round(time.Millisecond), a.stat("screenWatchdogRebuilds"), a.stat("screenWatchdogReshares"),
+ a.stat("screenWatchdogKeyframes")-keyframes, b.stat("screenReattaches"))
+ if !back {
+ t.Logf("a:\n%s", screenSections(a.diagnostics(t)))
+ t.Logf("b:\n%s", screenSections(b.diagnostics(t)))
+ t.Fatalf("b's wedged screen was not restored within 30 s: a=%+v b=%+v", a.sample(), b.sample())
+ }
+ if a.stat("screenWatchdogKeyframes") == keyframes {
+ t.Error("keyframes were not tried first")
+ }
+ if a.stat("screenWatchdogRebuilds") == 0 {
+ t.Error("b decoded again without a rebuild: the wedge did not hold")
+ }
+ waitNative(t, "b clears its report", func() bool { return b.stat("screenStalledHere") == 0 }, a, b)
+ waitNative(t, "a's notice clears", func() bool {
+ n := a.sample().ScreenRecovery
+ if n != "" {
+ notices[n] = true
+ }
+ return n == ""
+ }, a, b)
+ t.Logf("notices seen at a: %q", keys(notices))
+ if !anyHas(notices, "recovering") || !anyHas(notices, "Restarting your share for") {
+ t.Errorf("the notice did not move from recovering to restarting: %q", keys(notices))
+ }
+ watch(10*time.Second, "after the rebuild")
+ t.Logf("a's diagnostics:\n%s", screenSections(a.diagnostics(t)))
+ t.Logf("b's diagnostics:\n%s", screenSections(b.diagnostics(t)))
+ case "wedge-sticky":
+ // Every screen decoder B makes is wedged, the new ones too:
+ // nothing A does can mend it, and A has to say so rather than
+ // sit at "recovering".
+ watch(5*time.Second, "before")
+ b.command(t, "screen-decoder-wedge sticky")
+ notices := map[string]bool{}
+ gaveUp := false
+ for until := time.Now().Add(60 * time.Second); time.Now().Before(until); time.Sleep(250 * time.Millisecond) {
+ if n := a.sample().ScreenRecovery; n != "" {
+ notices[n] = true
+ }
+ if a.stat("screenWatchdogFailures") > 0 && anyHas(map[string]bool{a.sample().ScreenRecovery: true}, "Couldn\u2019t restore") {
+ gaveUp = true
+ break
+ }
+ }
+ t.Logf("notices seen at a: %q", keys(notices))
+ if !gaveUp {
+ t.Fatalf("a never said it could not restore b's view: a=%+v", a.sample())
+ }
+ if a.stat("screenWatchdogRebuilds") == 0 || a.stat("screenWatchdogReshares") == 0 {
+ t.Errorf("a gave up without trying a rebuild and a share again: %+v", a.sample().Stats)
+ }
+ if !anyHas(notices, "Restarting your share for") {
+ t.Errorf("the notice never said it was restarting: %q", keys(notices))
+ }
+ // The wedge lifted, the view comes back and the notice goes.
+ b.command(t, "screen-decoder-release")
+ recoveredTotal := func() bool {
+ base := b.stat("screenFramesDecodedTotal")
+ return pollNative(20*time.Second, func() bool { return b.stat("screenFramesDecodedTotal") > base+10 })
+ }
+ if !recoveredTotal() {
+ t.Fatalf("b did not decode again once the wedge was lifted: %+v", b.sample())
+ }
+ waitNative(t, "a's notice clears once b's view is back", func() bool { return a.sample().ScreenRecovery == "" }, a, b)
+ t.Logf("a's diagnostics:\n%s", screenSections(a.diagnostics(t)))
+ case "reshare":
+ // The watchdog's share again, on its own: every connection's
+ // screen stopped and added again from the same capture, and
+ // the watcher decoding again within a bound, one tile.
+ watch(5*time.Second, "before")
+ base := b.stat("screenFramesDecodedTotal")
+ started := time.Now()
+ a.command(t, "screen-reshare-test")
+ if !pollNative(15*time.Second, func() bool { return b.stat("screenFramesDecodedTotal") > base+30 }) {
+ t.Fatalf("b did not decode again after the share again: a=%+v b=%+v", a.sample(), b.sample())
+ }
+ t.Logf("b decoding again %v after the share again", time.Since(started).Round(time.Millisecond))
+ waitNative(t, "one screen at b", func() bool { return b.stat("remoteScreens") == 1 && len(b.sample().ScreenCaptions) == 1 }, a, b)
+ if a.sample().SharingScreen == nil || !*a.sample().SharingScreen {
+ t.Error("a stopped sharing")
+ }
+ watch(10*time.Second, "after the share again")
}
t.Logf("PASS %s: %d freezes, longest %v", scenario, freezes, longest)
})
}
}
+// screenSections is the screen's part of "Copy call diagnostics": the
+// "## Screen" sections and nothing else.
+func screenSections(diag string) string {
+ var out []string
+ in := false
+ for _, line := range strings.Split(diag, "\n") {
+ if strings.HasPrefix(line, "## ") {
+ in = strings.HasPrefix(line, "## Screen")
+ }
+ if in {
+ out = append(out, line)
+ }
+ }
+ return strings.Join(out, "\n")
+}
+
+func keys(m map[string]bool) []string {
+ out := make([]string, 0, len(m))
+ for k := range m {
+ out = append(out, k)
+ }
+ sort.Strings(out)
+ return out
+}
+
+func anyHas(m map[string]bool, part string) bool {
+ for k := range m {
+ if strings.Contains(k, part) {
+ return true
+ }
+ }
+ return false
+}
+
// pollNative is waitNative without the failure: whether ok held in time.
func pollNative(d time.Duration, ok func() bool) bool {
for until := time.Now().Add(d); time.Now().Before(until); time.Sleep(250 * time.Millisecond) {
diff --git a/native/README.md b/native/README.md
index 2f6f916..bfefb36 100644
--- a/native/README.md
+++ b/native/README.md
@@ -546,6 +546,43 @@ steps only if its own screen has stopped leaving too), and shows
**Copy audio diagnostics**). The apps restart ICE only when a relay credential is
due for renewal, not on every stream resume as 0.4.x did.
+**When keyframes are not enough** (spec §15.22, after Paul's "still seeing issues
+with screen sharing after a while", 30 Sep, the notice sitting at "recovering"):
+a receiver still reporting six seconds after the first keyframe gets its screen
+rebuilt — a new transceiver for that person alone, so a new encoder here and a new
+receiver and decoder there, at the same size and mode (a report never shrinks a
+Text and detail share) — and the notice says "Restarting your share for Samuel…".
+Still reported ten seconds later, everyone's screen is shared again from the same
+capture, with no picker: what Stop and Share did by hand. Ten seconds after that
+the notice says "Couldn't restore Samuel's view — try Stop and Share again". A
+rebuild at most once a minute per person, a share again at most once every five
+minutes, and neither unless frames actually left for that person since the last
+step. The receiver hands a stalled screen's picture to its view again four seconds
+in, logs what its statistics say is still arriving (packets, frames received,
+decoded, dropped, keyframes, PLI, the decoder), holds its report for up to three
+minutes, and counts any change in the decoded count as movement (a rebuilt screen's
+count starts again from zero, which 0.4.3 read as still stalled). The iPhone's
+broadcast repeats its last frame once a second while the screen is still, as the
+Mac's capture does, so a still iPhone screen is no longer reported as stalled. The
+browser does all of this too (its sharer's rebuild and share again included), and
+has **Copy call diagnostics** at the foot of its Devices panel.
+
+"Reconnecting…" is said only for a problem that lasts (spec §15.23): the
+signalling stream down 4 s, a connection `disconnected`/`failed` 3 s, or no
+audio for 2 s from someone unmuted — that one on their tile, "Reconnecting to
+Samuel…" — and it stays a second past the recovery. The stream's own renewal
+every few minutes is never said. `ReconnectWatch`; the `connection` log
+category.
+
+The gate for it wedges a receiver's screen decoder, which keyframes cannot mend:
+`screen-decoder-wedge` in `CallsInterop` (run with `CALLS_DECODER_WEDGE=1`; a
+wrapping decoder factory that only the gate turns on), and in Chromium an encoded
+transform that swallows every frame. `wedge`, `wedge-sticky` and `reshare` in
+`make native-pair`, and `wedge`, `wedge-sticky`, `web-wedge` and `web-wedge-sticky`
+in `make native-browser`, hold the ladder to restoring the view within 30 s, or to
+saying it cannot. `soak` and `web-soak` (`SCREEN_SOAK_SECONDS`) and the pair's
+`soak` (`SCREEN_PAIR_SOAK`) are the long shares, run by hand.
+
The numbers behind the modes come from `TestNativeMacScreenMotion` (in
`make native-browser`): the app shares a 1080p page scrolling at 30 fps with a
window dragged across it, and Chromium measures what arrives, with the app's
diff --git a/native/apple/Sources/CallsInterop/Interop.swift b/native/apple/Sources/CallsInterop/Interop.swift
index fe9013d..21ef045 100644
--- a/native/apple/Sources/CallsInterop/Interop.swift
+++ b/native/apple/Sources/CallsInterop/Interop.swift
@@ -56,6 +56,10 @@ struct Input: Decodable {
let input = try? JSONDecoder().decode(Input.self, from: Data(line.utf8)),
let url = URL(string: input.origin), url.scheme == "http", url.host == "127.0.0.1"
else { exit(2) }
+ // Decoders the gate can wedge (spec §15.22), set before the call's
+ // factory is made; only for the scenarios that ask, so every other
+ // gate decodes exactly as the apps do.
+ if ProcessInfo.processInfo.environment["CALLS_DECODER_WEDGE"] == "1" { DecoderWedge.enabled = true }
let call = Call(playAudio: false, audioInput: SyntheticVoice())
if ProcessInfo.processInfo.environment["CALLS_SCREEN_WATCHDOG"] == "0" { Call.screenWatchdogEnabled = false }
// WebRTC's own log, filtered to the video encoder, into a file: for
@@ -216,6 +220,34 @@ struct Input: Decodable {
print("{\"screenFrozenForTest\":true}")
fflush(stdout)
}
+ // §15.22: this app's screen decoders stop putting out pictures
+ // while packets keep arriving — a wedge no keyframe mends and a
+ // new transceiver does. `sticky` wedges the new ones too.
+ if command == "screen-decoder-wedge" || command == "screen-decoder-wedge sticky" {
+ let n = DecoderWedge.wedgeScreens(sticky: command.hasSuffix("sticky"))
+ print("{\"decoderWedged\":\(n)}")
+ fflush(stdout)
+ }
+ if command == "screen-decoder-release" {
+ DecoderWedge.release()
+ print("{\"decoderReleased\":true}")
+ fflush(stdout)
+ }
+ // "Copy call diagnostics", as the panel would copy it.
+ if command == "diagnostics" {
+ let text = await call.audioDiagnosticsText()
+ if let data = try? JSONSerialization.data(withJSONObject: ["diagnostics": text]),
+ let line = String(data: data, encoding: .utf8)
+ {
+ print(line)
+ fflush(stdout)
+ }
+ }
+ if command == "screen-reshare-test" {
+ await call.reshareScreenForTest()
+ print("{\"reshared\":true}")
+ fflush(stdout)
+ }
if command == "screen-end" {
await Interop.screen?.end()
let on = await call.sharingScreen
@@ -232,7 +264,9 @@ struct Input: Decodable {
// Long enough for the member scenario, screens and all, and for the
// motion gate's three modes: the gate always says `leave` long before
// this runs out.
- for _ in 0..<400 {
+ // CALLS_INTEROP_SECONDS for the soaks, which outlast that (§15.22).
+ let life = Int(ProcessInfo.processInfo.environment["CALLS_INTEROP_SECONDS"] ?? "") ?? 400
+ for _ in 0..<life {
try? await Task.sleep(nanoseconds: 1_000_000_000)
let stats = await call.sampleStats()
if let data = try? JSONSerialization.data(withJSONObject: [
@@ -249,6 +283,11 @@ struct Input: Decodable {
// place a screen says whose it is.
"screenCaptions": call.screens.map { screenCaption(call.name(of: $0)) },
"sharingScreen": call.sharingScreen,
+ // §15.22: the sharer's notice, word for word.
+ "screenRecovery": call.screenRecovery,
+ // §15.23: "Reconnecting…" said for the call, and on how many tiles.
+ "reconnecting": call.reconnecting,
+ "reconnectingPeers": call.reconnectingPeers.count,
"syntheticScreenFrames": Interop.screen?.produced ?? 0,
"cameraOpens": Interop.camera.opens,
"cameraStatus": {
@@ -285,6 +324,7 @@ struct Input: Decodable {
func stop() { mac.stop() }
var counts: ScreenCaptureCounts { mac.counts }
func restart() async -> Bool { await mac.restart() }
+ var frameSize: (width: Int, height: Int)? { mac.frameSize }
}
// Render the shipping chat view against real received messages. This fixture
diff --git a/native/apple/Sources/CallsInterop/SyntheticVideo.swift b/native/apple/Sources/CallsInterop/SyntheticVideo.swift
index 845c2c4..1ddf271 100644
--- a/native/apple/Sources/CallsInterop/SyntheticVideo.swift
+++ b/native/apple/Sources/CallsInterop/SyntheticVideo.swift
@@ -129,6 +129,7 @@ import LiveKitWebRTC
let n = page.map { UInt64($0.produced) } ?? made
return ScreenCaptureCounts(heartbeats: n, delivered: n)
}
+ var frameSize: (width: Int, height: Int)? { page != nil ? (ScrollingPage.width, ScrollingPage.height) : (640, 360) }
func restart() async -> Bool {
guard source != nil else { return false }
restarts += 1
@@ -186,7 +187,10 @@ import LiveKitWebRTC
/// each frame is row copies into a pooled buffer, on a queue of its own.
final class ScrollingPage: @unchecked Sendable {
static let width = 1920
- static let height = 1080
+ /// 1080 unless `CALLS_SYNTHETIC_SCREEN_HEIGHT` says otherwise: 1246 is a
+ /// 16:10 Retina display as the Mac's capture fits it to 1920 wide, the
+ /// size that meets H.264 level 3.1's block limit (spec §15.21, §15.22).
+ static let height = Int(ProcessInfo.processInfo.environment["CALLS_SYNTHETIC_SCREEN_HEIGHT"] ?? "") ?? 1080
private let rows = ScrollingPage.height * 2
private let texture: [UInt8]
private var offset = 0
diff --git a/native/apple/Sources/CallsKit/BroadcastScreenCapture.swift b/native/apple/Sources/CallsKit/BroadcastScreenCapture.swift
index 01ff1a8..0ce8d13 100644
--- a/native/apple/Sources/CallsKit/BroadcastScreenCapture.swift
+++ b/native/apple/Sources/CallsKit/BroadcastScreenCapture.swift
@@ -1,6 +1,7 @@
import CallsBroadcastKit
import CoreVideo
import Foundation
+import os
import LiveKitWebRTC
/// The iPhone's whole screen, arriving from the broadcast extension over
@@ -17,6 +18,7 @@ import LiveKitWebRTC
@MainActor public final class BroadcastScreenCapture: ScreenCapture {
private let connection: BroadcastConnection
private var stopped = false
+ private var feed: StillScreenFeed?
/// Called after the share has ended on its own, once the call has heard.
/// The app uses it to leave a call it is only still in for the screen.
public var afterEnded: (@MainActor () -> Void)?
@@ -24,17 +26,9 @@ import LiveKitWebRTC
public init(_ connection: BroadcastConnection) { self.connection = connection }
public func start(source: RTCVideoSource, ended: @escaping @MainActor () -> Void) async throws {
- let capturer = RTCVideoCapturer(delegate: source)
- connection.onFrame = { pixels, orientation in
- // Stamped on arrival, on this device's clock, rather than with the
- // extension's: the two are separate processes, and the encoder only
- // needs the frames in order and spaced as they came.
- source.capturer(
- capturer,
- didCapture: RTCVideoFrame(
- buffer: RTCCVPixelBuffer(pixelBuffer: pixels), rotation: broadcastRotation(orientation),
- timeStampNs: Int64(DispatchTime.now().uptimeNanoseconds)))
- }
+ let feed = StillScreenFeed(source: source)
+ self.feed = feed
+ connection.onFrame = { pixels, orientation in feed.frame(pixels, rotation: broadcastRotation(orientation)) }
connection.onClose = { [weak self] in
Task { @MainActor [weak self] in
guard let self, !self.stopped else { return }
@@ -46,8 +40,12 @@ import LiveKitWebRTC
connection.resume()
}
+ public var counts: ScreenCaptureCounts { feed?.counts ?? ScreenCaptureCounts() }
+
public func stop() {
stopped = true
+ feed?.stop()
+ feed = nil
connection.onClose = nil
connection.onFrame = nil
connection.close()
@@ -68,3 +66,67 @@ func broadcastRotation(_ orientation: UInt16) -> RTCVideoRotation {
default: return ._0
}
}
+
+/// The broadcast's frames into the call's source, and the last of them
+/// again once a second while the screen is still (2026-09-30). ReplayKit
+/// sends nothing at all for a screen that is not changing, and a receiver
+/// that sees no frame for four seconds reports the screen stalled (spec
+/// §15.21) — which, for a still iPhone screen, was a false alarm the
+/// sharer then tried to mend. The Mac's capture has repeated its last
+/// frame since 2026-09-28 (MacScreenCapture); this is the same rule.
+final class StillScreenFeed: @unchecked Sendable {
+ static let repeatAfter: UInt64 = 1_000_000_000
+
+ private let source: RTCVideoSource
+ private let capturer: RTCVideoCapturer
+ private let queue = DispatchQueue(label: "calls.broadcast.feed", qos: .userInteractive)
+ private var timer: DispatchSourceTimer?
+ // Only touched on `queue`.
+ private var last: (CVPixelBuffer, RTCVideoRotation)?
+ private var lastSent: UInt64 = 0
+ private let tally = OSAllocatedUnfairLock(initialState: ScreenCaptureCounts(heartbeats: nil, delivered: 0))
+ var counts: ScreenCaptureCounts { tally.withLock { $0 } }
+
+ init(source: RTCVideoSource) {
+ self.source = source
+ capturer = RTCVideoCapturer(delegate: source)
+ let timer = DispatchSource.makeTimerSource(queue: queue)
+ timer.schedule(deadline: .now() + .milliseconds(500), repeating: .milliseconds(500), leeway: .milliseconds(50))
+ timer.setEventHandler { [weak self] in self?.repeatIfIdle() }
+ timer.resume()
+ self.timer = timer
+ }
+
+ /// One frame from the extension. Stamped on arrival, on this device's
+ /// clock, rather than with the extension's: the two are separate
+ /// processes, and the encoder only needs the frames in order and spaced
+ /// as they came.
+ func frame(_ pixels: CVPixelBuffer, rotation: RTCVideoRotation) {
+ queue.async {
+ self.last = (pixels, rotation)
+ self.deliver(pixels, rotation)
+ }
+ }
+
+ func stop() {
+ timer?.cancel()
+ timer = nil
+ queue.async { self.last = nil }
+ }
+
+ private func repeatIfIdle() {
+ guard let (pixels, rotation) = last else { return }
+ let now = DispatchTime.now().uptimeNanoseconds
+ guard now >= lastSent, now - lastSent >= Self.repeatAfter else { return }
+ deliver(pixels, rotation)
+ }
+
+ private func deliver(_ pixels: CVPixelBuffer, _ rotation: RTCVideoRotation) {
+ let ns = max(DispatchTime.now().uptimeNanoseconds, lastSent + 1)
+ lastSent = ns
+ tally.withLock { $0.delivered += 1 }
+ source.capturer(
+ capturer,
+ didCapture: RTCVideoFrame(buffer: RTCCVPixelBuffer(pixelBuffer: pixels), rotation: rotation, timeStampNs: Int64(ns)))
+ }
+}
diff --git a/native/apple/Sources/CallsKit/Call.swift b/native/apple/Sources/CallsKit/Call.swift
index 80d24aa..56f4107 100644
--- a/native/apple/Sources/CallsKit/Call.swift
+++ b/native/apple/Sources/CallsKit/Call.swift
@@ -75,6 +75,26 @@ private final class Events: NSObject, BridgeListenerProtocol {
/// `screenStalled` (spec §15.21).
private var screenReceiveWatch = ScreenReceiveWatch()
private var reportedStalled: [String] = []
+ /// Each arriving screen's statistics at the last look, for the stall
+ /// log's "what moved" and for "Copy call diagnostics".
+ private var screenReceived: [String: ScreenReceiveReport] = [:]
+ /// Where each arriving screen stood when it last moved, so a stall can
+ /// be described against it.
+ private var screenReceivedMoving: [String: ScreenReceiveReport] = [:]
+ private var screenReattaches = 0
+ /// "Reconnecting…", said only for a problem that lasts (spec §15.23):
+ /// the whole call's, and the people whose tiles say it on their own.
+ private var reconnectWatch = ReconnectWatch()
+ @Published public private(set) var reconnecting = false
+ @Published public private(set) var reconnectingPeers: Set<String> = []
+ /// The phase the notices are drawn for: the core's, except that its
+ /// `reconnecting` is only said once ReconnectWatch says so, and is said
+ /// for a second after it is over.
+ public var noticePhase: String? {
+ if reconnecting { return "reconnecting" }
+ if state?.phase == "reconnecting" { return "admitted" }
+ return state?.phase
+ }
private var screenWatchFrozenUntil = 0.0
@Published public private(set) var localTrack: RTCVideoTrack?
@Published public private(set) var muted = false
@@ -202,11 +222,28 @@ private final class Events: NSObject, BridgeListenerProtocol {
} else {
lines.append("Not sharing")
}
+ if sharingScreen {
+ for peer in (state?.peers ?? []).map(\.peer) where screenWatchdog.escalation(of: peer) != .none {
+ lines.append("recovering \(name(of: peer))'s view: \(screenWatchdog.escalation(of: peer))")
+ }
+ lines.append(
+ "recoveries: \(screenWatchdog.keyframes) keyframes, \(screenWatchdog.encoderRecoveries) fresh encoders, \(screenWatchdog.reAdds) re-adds, \(screenWatchdog.rebuilds) rebuilds, \(screenWatchdog.reshares) shares again, \(screenWatchdog.failures) given up")
+ }
+ // The receiver's half: every screen arriving here, as its statistics
+ // have it, whether or not it has stalled.
+ for (peer, report) in screenReceived.sorted(by: { $0.key < $1.key }) {
+ lines.append("from \(name(of: peer)): \(report.summary)")
+ }
if !reportedStalled.isEmpty {
lines.append("stalled here: " + reportedStalled.map { name(of: $0) }.joined(separator: ", "))
}
- let log = AudioDiagnosticsStore.screen.lines(last: 50)
- return [("Screen", lines), ("Screen log (last 50 lines)", log.isEmpty ? ["(nothing logged)"] : log)]
+ if screenReattaches > 0 { lines.append("pictures attached again here: \(screenReattaches)") }
+ let log = AudioDiagnosticsStore.screen.lines(last: AudioDiagnosticsStore.lineLimit)
+ let connection = AudioDiagnosticsStore.connection.lines(last: 50)
+ return [
+ ("Screen", lines), ("Screen log (last \(AudioDiagnosticsStore.lineLimit) lines)", log.isEmpty ? ["(nothing logged)"] : log),
+ ("Connection log (last 50 lines)", connection.isEmpty ? ["(nothing logged)"] : connection),
+ ]
}
private var ice: ICEConfig?
private var pendingMedia: [Message] = []
@@ -504,6 +541,8 @@ private final class Events: NSObject, BridgeListenerProtocol {
levelWatch = nil
screenReceiveWatch = ScreenReceiveWatch()
reportedStalled = []
+ screenReceived = [:]
+ screenReceivedMoving = [:]
levels.reset()
reportedNoSound = []
audioBytes = [:]
@@ -540,6 +579,7 @@ private final class Events: NSObject, BridgeListenerProtocol {
guard levelWatch != nil, let state else { return }
let me = state.peer
var present = Set<String>()
+ var people: [String: ReconnectWatch.Peer] = [:]
for person in state.peers where person.peer != me {
guard let reading = readings[person.peer] else { continue }
present.insert(person.peer)
@@ -547,15 +587,21 @@ private final class Events: NSObject, BridgeListenerProtocol {
audioArriving[person.peer] = bytes > (audioBytes[person.peer] ?? 0)
audioBytes[person.peer] = bytes
}
+ people[person.peer] = ReconnectWatch.Peer(
+ connectionBad: reading.connectionBad,
+ audioMissing: audioArriving[person.peer] == false && person.media?.muted != true)
+ var notices = AudioWords.notices(name: person.name, media: person.media, me: me, muted: muted)
+ if reconnectingPeers.contains(person.peer) { notices.insert(ReconnectWords.peer(person.name), at: 0) }
remotePeakSinceSample = max(remotePeakSinceSample, reading.peak)
levels.updatePeer(
person.peer, name: person.name,
sample: PeerAudioSample(
reported: person.media, connected: reading.connected,
arriving: audioArriving[person.peer] ?? false, level: reading.peak),
- notices: AudioWords.notices(name: person.name, media: person.media, me: me, muted: muted),
+ notices: notices,
now: now)
}
+ watchReconnect(now: now, people: people)
levels.settle(present: present)
for id in Array(audioBytes.keys) where !present.contains(id) {
audioBytes.removeValue(forKey: id)
@@ -564,6 +610,23 @@ private final class Events: NSObject, BridgeListenerProtocol {
reportNoSound(levels.noSoundFrom)
}
+ /// One look by ReconnectWatch (spec §15.23), and the words it says.
+ private func watchReconnect(now: Double, people: [String: ReconnectWatch.Peer]) {
+ reconnectWatch.observe(at: now, streamDown: state?.phase == "reconnecting", people: people)
+ if reconnectingPeers != reconnectWatch.peers {
+ for id in reconnectWatch.peers.subtracting(reconnectingPeers) { CallLog.info("reconnecting to \(id.prefix(6)): shown") }
+ reconnectingPeers = reconnectWatch.peers
+ }
+ guard reconnecting != reconnectWatch.call else { return }
+ reconnecting = reconnectWatch.call
+ CallLog.info(reconnecting ? "reconnecting: shown (a problem that lasted)" : "reconnecting: cleared")
+ switch state?.phase {
+ case "reconnecting": status = reconnecting ? ReconnectWords.call : "In the meeting"
+ case "admitted": status = reconnecting ? ReconnectWords.call : "In the meeting"
+ default: break
+ }
+ }
+
/// §15.20's receiving half: who is reaching this app as silence. The
/// core keeps only roster peers and reports on a change; this sends only
/// on a change too, in order, off the main thread.
@@ -640,10 +703,12 @@ private final class Events: NSObject, BridgeListenerProtocol {
KnockNotice.announce(
knocks.arrivals(waiting: state.waiting, visible: KnockNotice.appIsFrontmost))
switch state.phase {
- case "admitted": status = "In the meeting"
+ case "admitted": status = reconnecting ? ReconnectWords.call : "In the meeting"
case "waiting": status = "Waiting to be let in"
case "full": status = "The meeting is full. Try joining again shortly."
- case "reconnecting": status = "Reconnecting…"
+ // Said only once it has lasted (ReconnectWatch, spec §15.23): a
+ // stream renewal is a reconnect of half a second nobody should see.
+ case "reconnecting": if reconnecting { status = ReconnectWords.call }
case "ended":
status =
[
@@ -823,20 +888,41 @@ private final class Events: NSObject, BridgeListenerProtocol {
case .reAdd(let peer):
ScreenLog.info("screen re-added for \(ScreenWatchdog.short(peer))")
media.reAddScreen(peer)
- // The new transceiver's track arrives unlabelled; say it again.
- Task { @MainActor [weak self] in
- try? await Task.sleep(nanoseconds: 2_000_000_000)
- guard let self, self.sharingScreen else { return }
- self.send(type: "share", to: "", gen: 0, payload: json(["on": true]) ?? "")
- }
+ sayShareAgain()
+ case .rebuild(let peer):
+ ScreenLog.info("screen rebuilt for \(ScreenWatchdog.short(peer)): new transceiver, same size and mode")
+ media.reAddScreen(peer)
+ sayShareAgain()
+ case .reshare:
+ ScreenLog.info("sharing again for everyone, from the same capture")
+ await media.reshareScreen()
+ sayShareAgain()
}
}
screenWatchStatus = screenWatchdog.status
if let (peer, worst) = reports.min(by: { $0.value.framesPerSecond < $1.value.framesPerSecond }) {
screenDetail = worst.detail(since: previousReports[peer], watchdog: screenWatchdog.status)
}
- let names = stalledBy.map { name(of: $0) }.sorted()
- screenRecovery = ScreenWords.recovering(names: names)
+ var recovering: [String] = []
+ var restarting: [String] = []
+ var failed: [String] = []
+ for peer in stalledBy.sorted(by: { name(of: $0) < name(of: $1) }) {
+ switch screenWatchdog.escalation(of: peer) {
+ case .rebuilt, .reshared: restarting.append(name(of: peer))
+ case .failed: failed.append(name(of: peer))
+ case .none, .keyframes: recovering.append(name(of: peer))
+ }
+ }
+ screenRecovery = ScreenWords.notice(recovering: recovering, restarting: restarting, failed: failed)
+ }
+
+ /// A new transceiver's track arrives unlabelled; say it again.
+ private func sayShareAgain() {
+ Task { @MainActor [weak self] in
+ try? await Task.sleep(nanoseconds: 2_000_000_000)
+ guard let self, self.sharingScreen else { return }
+ self.send(type: "share", to: "", gen: 0, payload: json(["on": true]) ?? "")
+ }
}
private func endScreenWatch() {
screenWatch?.cancel()
@@ -851,30 +937,67 @@ private final class Events: NSObject, BridgeListenerProtocol {
/// The receiver's watchdog, once a second from the level watch: whose
/// screens have stopped moving here, reported as `screenStalled` when
- /// that changes (spec §15.21).
+ /// that changes (spec §15.21), each stall logged with what the
+ /// statistics say is still arriving, and the picture attached again
+ /// four seconds in (spec §15.22).
private func watchRemoteScreens(now: Double) async {
var stalled: [String] = []
- if !remoteScreens.isEmpty {
- var frames = await media.remoteScreenFrames()
+ if !screens.isEmpty {
+ let reports = await media.remoteScreenReports()
+ var frames = reports.mapValues(\.framesDecoded)
// The gate's frozen picture: the counts held where they were.
if now < screenWatchFrozenUntil { frames = frames.mapValues { _ in 0 } }
let sharing = Set(screens)
+ let before = screenReceived
+ screenReceived = reports.filter { sharing.contains($0.key) }
stalled = screenReceiveWatch.observe(at: now, decoded: frames.filter { sharing.contains($0.key) })
+ for (peer, report) in screenReceived where !stalled.contains(peer) {
+ if before[peer].map({ $0.framesDecoded != report.framesDecoded || $0.mid != report.mid }) ?? true {
+ screenReceivedMoving[peer] = report
+ }
+ }
+ for peer in stalled where !reportedStalled.contains(peer) {
+ let report = screenReceived[peer] ?? ScreenReceiveReport()
+ let moving = screenReceivedMoving[peer]
+ ScreenLog.info(
+ "screen from \(ScreenWatchdog.short(peer)) stalled here: reporting it; since it last moved \(report.moved(since: moving)) — \(report.diagnosis(since: moving))")
+ ScreenLog.info("screen from \(ScreenWatchdog.short(peer)): \(report.summary)")
+ }
+ for peer in screenReceiveWatch.reattach {
+ let report = screenReceived[peer] ?? ScreenReceiveReport()
+ ScreenLog.info(
+ "screen from \(ScreenWatchdog.short(peer)) still stalled: attaching its picture again; \(report.moved(since: screenReceivedMoving[peer]))")
+ reattachScreen(peer)
+ }
} else {
screenReceiveWatch = ScreenReceiveWatch()
+ screenReceived = [:]
+ screenReceivedMoving = [:]
}
guard stalled != reportedStalled, let client = bridge else { return }
- for peer in stalled where !reportedStalled.contains(peer) {
- ScreenLog.info("screen from \(ScreenWatchdog.short(peer)) stalled here: asking for a keyframe")
- }
for peer in reportedStalled where !stalled.contains(peer) {
- ScreenLog.info("screen from \(ScreenWatchdog.short(peer)) moving here again")
+ let report = screenReceived[peer]
+ ScreenLog.info(
+ "screen from \(ScreenWatchdog.short(peer)) moving here again\(report.map { ": " + $0.summary } ?? "")")
}
reportedStalled = stalled
let body = json(stalled) ?? "[]"
noSoundQueue.async { try? client.setScreenStalled(body) }
}
+ /// A stalled screen's picture dropped and attached again: a new renderer
+ /// on the same track, for a picture stopped on this side.
+ private func reattachScreen(_ peer: String) {
+ guard let track = remoteScreens[peer] else { return }
+ screenReattaches += 1
+ remoteScreens.removeValue(forKey: peer)
+ Task { @MainActor [weak self] in
+ try? await Task.sleep(nanoseconds: 150_000_000)
+ guard let self, self.screens.contains(peer), self.remoteScreens[peer] == nil else { return }
+ self.remoteScreens[peer] = track
+ }
+ }
+
/// Stops sharing, from every way out: the button, the system's own stop
/// control, and the capture ending under us.
public func stopSharingScreen() {
@@ -915,6 +1038,13 @@ private final class Events: NSObject, BridgeListenerProtocol {
}
/// For the gate's encoder stall (see MediaEngine).
@_spi(Fixtures) public func stallScreenEncoderForTest() { media.stallScreenEncoderForTest() }
+ /// For the gate: the watchdog's share again (spec §15.22), on demand.
+ @_spi(Fixtures) public func reshareScreenForTest() async {
+ guard sharingScreen else { return }
+ ScreenLog.info("sharing again for everyone, from the same capture (asked for by the gate)")
+ await media.reshareScreen()
+ sayShareAgain()
+ }
public func sampleStats() async -> [String: UInt64] {
var values = await media.stats()
values["hasICE"] = ice == nil ? 0 : 1
@@ -952,6 +1082,10 @@ private final class Events: NSObject, BridgeListenerProtocol {
values["screenWatchdogReAdds"] = UInt64(screenWatchdog.reAdds)
values["screenStallReports"] = UInt64(screenWatchdog.stallReports)
values["screenStalledHere"] = UInt64(reportedStalled.count)
+ values["screenWatchdogRebuilds"] = UInt64(screenWatchdog.rebuilds)
+ values["screenWatchdogReshares"] = UInt64(screenWatchdog.reshares)
+ values["screenWatchdogFailures"] = UInt64(screenWatchdog.failures)
+ values["screenReattaches"] = UInt64(screenReattaches)
values["iceRestarts"] = media.iceRestarts
values["screenKeyframes"] = screenReport?.keyFramesEncoded ?? 0
if let capture = media.screenCaptureCounts {
@@ -1006,6 +1140,8 @@ extension Call {
self.standing = standing
self.state = try? JSONDecoder().decode(RoomState.self, from: Data(json.utf8))
self.status = status
+ // A fixture's reconnecting is one that has lasted (spec §15.23).
+ self.reconnecting = self.state?.phase == "reconnecting"
self.screens = screens
self.sharingScreen = sharingScreen
self.muted = muted
diff --git a/native/apple/Sources/CallsKit/CallAudio.swift b/native/apple/Sources/CallsKit/CallAudio.swift
index f3d53df..dd281b5 100644
--- a/native/apple/Sources/CallsKit/CallAudio.swift
+++ b/native/apple/Sources/CallsKit/CallAudio.swift
@@ -158,7 +158,9 @@ public struct ProcessingReport: Equatable, Sendable {
denoiser: CaptureDenoiser?
) -> Parts {
let encoder = RTCDefaultVideoEncoderFactory()
- let decoder = RTCDefaultVideoDecoderFactory()
+ // WebRTC's own, except in the gate's decoder-wedge scenarios.
+ let decoder: RTCVideoDecoderFactory =
+ DecoderWedge.enabled ? DecoderWedge.factory() : RTCDefaultVideoDecoderFactory()
let denoiser = denoiser ?? CaptureDenoiser(suppressor: CallAudio.makeSuppressor())
#if os(iOS)
if let audioDevice {
diff --git a/native/apple/Sources/CallsKit/Capture.swift b/native/apple/Sources/CallsKit/Capture.swift
index 3b773c2..15ec995 100644
--- a/native/apple/Sources/CallsKit/Capture.swift
+++ b/native/apple/Sources/CallsKit/Capture.swift
@@ -104,11 +104,18 @@ extension VideoCapture {
/// source — after the watchdog has seen it stall. False when this kind
/// of capture cannot, or the attempt failed.
func restart() async -> Bool
+ /// The size the capture delivers frames at, when it knows before the
+ /// first frame: the send policy's H.264 level cap (ScreenSendPolicy)
+ /// then holds from the first frame rather than from the first
+ /// statistics a second later, a second in which a browser's level-3.1
+ /// encoder refuses every frame of a Motion share (spec §15.22).
+ var frameSize: (width: Int, height: Int)? { get }
}
extension ScreenCapture {
public var counts: ScreenCaptureCounts { ScreenCaptureCounts() }
public func restart() async -> Bool { false }
+ public var frameSize: (width: Int, height: Int)? { nil }
}
/// A screen capture's running totals (see ScreenWatchdog).
diff --git a/native/apple/Sources/CallsKit/ChatView.swift b/native/apple/Sources/CallsKit/ChatView.swift
index 8f99bd6..814b481 100644
--- a/native/apple/Sources/CallsKit/ChatView.swift
+++ b/native/apple/Sources/CallsKit/ChatView.swift
@@ -35,7 +35,7 @@ import SwiftUI
proxy.scrollTo("chat-bottom", anchor: .bottom)
}
}
- if call.state?.phase == "reconnecting" {
+ if call.reconnecting {
Text("Reconnecting. You can send again when the call reconnects.")
.font(.footnote).foregroundStyle(.secondary)
}
diff --git a/native/apple/Sources/CallsKit/DecoderWedge.swift b/native/apple/Sources/CallsKit/DecoderWedge.swift
new file mode 100644
index 0000000..bb775a0
--- /dev/null
+++ b/native/apple/Sources/CallsKit/DecoderWedge.swift
@@ -0,0 +1,123 @@
+import Foundation
+import LiveKitWebRTC
+import os
+
+/// For the gate only (spec §15.22): a receiver whose screen decoder has
+/// wedged. Packets keep arriving, frames keep being handed to the decoder,
+/// and nothing comes out — which no keyframe mends, because it is this
+/// decoder instance that is stuck, and which a new transceiver does mend,
+/// because the far end's new stream gets a decoder of its own. That is the
+/// shape of a stall that "keyframes only" (0.4.3) left sitting at
+/// "recovering".
+///
+/// Off unless the interop executable turns it on before the call is
+/// built; the apps never do, and then the call's factory is WebRTC's own.
+@_spi(Fixtures) public enum DecoderWedge {
+ /// Wrap the call's decoders so they can be wedged. Read once, when the
+ /// call's factory is made.
+ nonisolated(unsafe) public static var enabled = false
+
+
+ private static let live = OSAllocatedUnfairLock(initialState: [WeakDecoder]())
+ private static let sticky = OSAllocatedUnfairLock(initialState: false)
+
+ static func register(_ decoder: WedgeableDecoder) {
+ live.withLock { list in
+ list.removeAll { $0.decoder == nil }
+ list.append(WeakDecoder(decoder: decoder))
+ }
+ }
+
+ /// Wedges the decoder now putting out the largest picture, which in the
+ /// gate is always the screen (its cameras are 320×240 to 640×480, and a
+ /// screen even at its smallest under a thin link is wider). `sticky`
+ /// wedges every decoder made afterwards too — only a screen's rebuild or
+ /// share again makes one mid-call — for a receiver no rebuild can mend,
+ /// the gate's "could not restore". Returns how many were wedged.
+ @discardableResult public static func wedgeScreens(sticky: Bool = false) -> Int {
+ self.sticky.withLock { $0 = sticky }
+ let decoders = live.withLock { $0.compactMap(\.decoder) }
+ guard let largest = decoders.map(\.area).max(), largest > 0 else { return 0 }
+ var count = 0
+ for d in decoders where d.area == largest {
+ d.wedge()
+ count += 1
+ }
+ return count
+ }
+
+ /// Every wedge lifted, and nothing wedged from now on.
+ public static func release() {
+ sticky.withLock { $0 = false }
+ for d in live.withLock({ $0.compactMap(\.decoder) }) { d.unwedge() }
+ }
+
+ static var isSticky: Bool { sticky.withLock { $0 } }
+
+ static func factory() -> RTCVideoDecoderFactory { WedgeableDecoderFactory() }
+}
+
+private struct WeakDecoder {
+ weak var decoder: WedgeableDecoder?
+}
+
+final class WedgeableDecoderFactory: NSObject, RTCVideoDecoderFactory {
+ private let inner = RTCDefaultVideoDecoderFactory()
+
+ func createDecoder(_ info: RTCVideoCodecInfo) -> RTCVideoDecoder? {
+ guard let decoder = inner.createDecoder(info) else { return nil }
+ // Only the VideoToolbox decoders are Objective-C all the way down.
+ // WebRTC's own VP8, VP9 and AV1 come back as wrapped native decoders,
+ // which the factory unwraps and never calls through this protocol, so
+ // they are handed over as they are and cannot be wedged.
+ guard ["H264", "H265"].contains(info.name.uppercased()) else { return decoder }
+ let wrapped = WedgeableDecoder(decoder)
+ if DecoderWedge.isSticky { wrapped.wedge() }
+ DecoderWedge.register(wrapped)
+ return wrapped
+ }
+
+ func supportedCodecs() -> [RTCVideoCodecInfo] { inner.supportedCodecs() }
+}
+
+final class WedgeableDecoder: NSObject, RTCVideoDecoder, @unchecked Sendable {
+ private let inner: RTCVideoDecoder
+ private struct State {
+ var wedged = false
+ var area = 0
+ }
+ private let state = OSAllocatedUnfairLock(initialState: State())
+
+ init(_ inner: RTCVideoDecoder) { self.inner = inner }
+
+ var area: Int { state.withLock { $0.area } }
+ func wedge() { state.withLock { $0.wedged = true } }
+ func unwedge() { state.withLock { $0.wedged = false } }
+
+ func setCallback(_ callback: @escaping RTCVideoDecoderCallback) {
+ inner.setCallback { [weak self] frame in
+ guard let self else { return }
+ let area = Int(frame.width) * Int(frame.height)
+ let wedged = self.state.withLock { s -> Bool in
+ s.area = area
+ return s.wedged
+ }
+ if !wedged { callback(frame) }
+ }
+ }
+
+ func startDecode(withNumberOfCores numberOfCores: Int32) -> Int {
+ inner.startDecode(withNumberOfCores: numberOfCores)
+ }
+
+ func release() -> Int { inner.release() }
+
+ func decode(
+ _ encodedImage: RTCEncodedImage, missingFrames: Bool, codecSpecificInfo info: RTCCodecSpecificInfo?,
+ renderTimeMs: Int64
+ ) -> Int {
+ inner.decode(encodedImage, missingFrames: missingFrames, codecSpecificInfo: info, renderTimeMs: renderTimeMs)
+ }
+
+ func implementationName() -> String { inner.implementationName() }
+}
diff --git a/native/apple/Sources/CallsKit/MacFullWindowCall.swift b/native/apple/Sources/CallsKit/MacFullWindowCall.swift
index 1e79c08..cdfc3f6 100644
--- a/native/apple/Sources/CallsKit/MacFullWindowCall.swift
+++ b/native/apple/Sources/CallsKit/MacFullWindowCall.swift
@@ -304,7 +304,7 @@
private var notices: [CallNotice] {
CallNotices.notices(
- phase: call.state?.phase, status: call.status, sharingScreen: call.sharingScreen,
+ phase: call.noticePhase, status: call.status, sharingScreen: call.sharingScreen,
screenNote: call.screenNote, floor: call.floor, screenRecovery: call.screenRecovery)
}
diff --git a/native/apple/Sources/CallsKit/MacScreenCapture.swift b/native/apple/Sources/CallsKit/MacScreenCapture.swift
index 2ee3714..b5b38d8 100644
--- a/native/apple/Sources/CallsKit/MacScreenCapture.swift
+++ b/native/apple/Sources/CallsKit/MacScreenCapture.swift
@@ -108,7 +108,10 @@
self.frames = frames
self.filter = filter
self.source = source
+ let configured = Self.configuration(for: filter)
+ frameSize = (configured.width, configured.height)
}
+ public private(set) var frameSize: (width: Int, height: Int)?
/// The same content into the same source, on a capture of its own:
/// a fresh stream with a fresh pool of surfaces. What stopping and
diff --git a/native/apple/Sources/CallsKit/Media.swift b/native/apple/Sources/CallsKit/Media.swift
index 609fe81..92857a4 100644
--- a/native/apple/Sources/CallsKit/Media.swift
+++ b/native/apple/Sources/CallsKit/Media.swift
@@ -139,9 +139,32 @@ import LiveKitWebRTC
throw CancellationError()
}
screenCapture = capture
+ screenSourceInUse = source
+ if let size = capture.frameSize, size.width > 0, size.height > 0 { screenSize = size }
localScreen = factory.videoTrack(with: source, trackId: "screen")
for peer in peers.values { peer.addScreen() }
}
+ /// The source the screen's capture is writing into, for a share again
+ /// from the same capture.
+ private var screenSourceInUse: RTCVideoSource?
+ private var reshares = 0
+
+ /// Stop and Share, without the picker (spec §15.22): every connection's
+ /// screen transceiver stopped and renegotiated, a fresh track on the
+ /// same source, the capture started again on the same content, and each
+ /// connection given the new track as soon as it settles. New
+ /// transceivers mean new encoders here and new receivers and decoders at
+ /// every far end, which is what the hand-made stop and share did.
+ func reshareScreen() async {
+ guard let capture = screenCapture, let source = screenSourceInUse, localScreen != nil else { return }
+ reshares += 1
+ for peer in peers.values { peer.removeScreen() }
+ localScreen = factory.videoTrack(with: source, trackId: "screen-\(reshares)")
+ if !(await capture.restart()) { ScreenLog.info("share again: the capture carries on as it was") }
+ guard screenCapture === capture else { return }
+ // Any connection already stable takes it now; the rest as they settle.
+ for peer in peers.values { peer.addScreen() }
+ }
/// Stops sending the screen: every connection's screen transceiver is
/// stopped and renegotiated, so the receivers' tracks *end*, and the
@@ -150,6 +173,7 @@ import LiveKitWebRTC
screenAttempt += 1
guard let capture = screenCapture else { return }
screenCapture = nil
+ screenSourceInUse = nil
localScreen = nil
for peer in peers.values {
peer.removeScreen()
@@ -181,24 +205,36 @@ import LiveKitWebRTC
for peer in peers.values { peer.setScreenActive(false) }
}
- /// Frames decoded so far from each peer's screen, for the peers whose
- /// screen is arriving here (the receiver's watchdog).
- func remoteScreenFrames() async -> [String: UInt64] {
- var out: [String: UInt64] = [:]
+ /// Each peer's screen as it arrives here, for the peers whose screen is
+ /// arriving (the receiver's watchdog, and "Copy call diagnostics").
+ func remoteScreenReports() async -> [String: ScreenReceiveReport] {
+ var out: [String: ScreenReceiveReport] = [:]
for (id, peer) in peers {
guard let pc = peer.pc, peer.receivingScreen else { continue }
let report = await PeerCalls.statistics(pc)
- var decoded: UInt64 = 0
+ var reading = ScreenReceiveReport()
for stat in report.statistics.values where stat.type == "inbound-rtp" {
let mid = stat.values["mid"] as? String ?? ""
guard peer.isScreen(mid: mid) else { continue }
- decoded += (stat.values["framesDecoded"] as? NSNumber)?.uint64Value ?? 0
+ reading.add(stat.values)
}
- out[id] = decoded
+ reading.mid = peer.screenMidHere
+ out[id] = reading
}
return out
}
+ /// Frames decoded from screens here, never going backwards: a rebuilt
+ /// screen arrives on a new receiver whose own count starts again, and
+ /// the gate's "decoding again" has to see its frames as more.
+ private var screenDecodedSeen: [String: UInt64] = [:]
+ private(set) var screenDecodedTotal: UInt64 = 0
+ private func tallyScreenDecoded(_ peer: String, _ now: UInt64) {
+ let last = screenDecodedSeen[peer] ?? 0
+ screenDecodedTotal += now >= last ? now - last : now
+ screenDecodedSeen[peer] = now
+ }
+
/// Puts every connection's screen and camera senders where the policy
/// wants them; a no-op for any that already are.
func applySendPolicy() {
@@ -296,10 +332,12 @@ import LiveKitWebRTC
"insertedSamplesForDeceleration": 0, "removedSamplesForAcceleration": 0,
"jitterBufferDelayMs": 0,
]
- for peer in peers.values {
+ for (id, peer) in peers {
guard let pc = peer.pc else { continue }
let ownScreen = await peer.ownScreenMid()
let report = await PeerCalls.statistics(pc)
+ var screenDecoded: UInt64 = 0
+ defer { tallyScreenDecoded(id, screenDecoded) }
for stat in report.statistics.values {
let kind = (stat.values["kind"] as? String) ?? (stat.values["mediaType"] as? String) ?? ""
let mid = stat.values["mid"] as? String ?? ""
@@ -310,8 +348,9 @@ import LiveKitWebRTC
// for a camera that had stopped. Told apart by mid, which is
// what the browser's own gate does.
let field = peer.isScreen(mid: mid) ? "screenFramesDecoded" : "framesDecoded"
- values[field, default: 0] +=
- (stat.values["framesDecoded"] as? NSNumber)?.uint64Value ?? 0
+ let decoded = (stat.values["framesDecoded"] as? NSNumber)?.uint64Value ?? 0
+ values[field, default: 0] += decoded
+ if peer.isScreen(mid: mid) { screenDecoded += decoded }
}
if kind == "audio" {
values["audioBytesReceived", default: 0] +=
@@ -341,6 +380,7 @@ import LiveKitWebRTC
}
}
}
+ values["screenFramesDecodedTotal"] = screenDecodedTotal
return values
}
@@ -361,7 +401,8 @@ import LiveKitWebRTC
(pc.iceConnectionState, stats ? pc.receivers.filter { $0.track?.kind == "audio" } : [])
}
var reading = PeerAudioReading(
- peak: peak, buffers: buffers, connected: state == .connected || state == .completed)
+ peak: peak, buffers: buffers, connected: state == .connected || state == .completed,
+ connectionBad: state == .disconnected || state == .failed)
if stats {
var bytes: UInt64 = 0
for receiver in receivers {
@@ -421,6 +462,8 @@ struct PeerAudioReading: Equatable, Sendable {
var peak: Float
var buffers: Int
var connected: Bool
+ /// ICE `disconnected` or `failed` (spec §15.23's three seconds).
+ var connectionBad = false
var bytes: UInt64?
}
@@ -640,6 +683,7 @@ struct PeerAudioReading: Equatable, Sendable {
}
}
func isScreen(mid: String) -> Bool { !mid.isEmpty && mid == screenMid }
+ var screenMidHere: String { screenMid }
/// The mid our screen went out on, once negotiated: read on the audio
/// queue (a transceiver's mid is a hop to WebRTC's signalling thread),
/// and kept once known.
diff --git a/native/apple/Sources/CallsKit/ReconnectWatch.swift b/native/apple/Sources/CallsKit/ReconnectWatch.swift
new file mode 100644
index 0000000..023c902
--- /dev/null
+++ b/native/apple/Sources/CallsKit/ReconnectWatch.swift
@@ -0,0 +1,125 @@
+import Foundation
+import os
+
+/// When "Reconnecting…" is said, and to whom (spec §15.23).
+///
+/// Paul, 2026-09-30: "During that call I also saw 'Reconnecting' appear
+/// occasionally over the video, but only for a second … I didn't notice any
+/// interruption in the call." The core puts the call in its `reconnecting`
+/// phase the moment its signalling stream ends, and the stream ends on
+/// purpose every few minutes (its renewal) and is resumed half a second
+/// later; the media never notices. So the words wait until a problem has
+/// lasted:
+///
+/// - the signalling stream down for `streamAfter` (4 s) — the whole call;
+/// - a connection `disconnected` or `failed` for `connectionAfter` (3 s),
+/// or no audio arriving from someone who is not muted for `audioAfter`
+/// (2 s) — that person's tile, "Reconnecting to Samuel…", unless it is
+/// everyone (two or more), which is the whole call again.
+///
+/// Once said, the words stay `holdAfter` (1 s) past the recovery, so they
+/// do not flicker. Pure: fed the facts as they are, with the time.
+struct ReconnectWatch: Equatable {
+ static let streamAfter = 4.0
+ static let connectionAfter = 3.0
+ static let audioAfter = 2.0
+ static let holdAfter = 1.0
+
+ /// One person's connection as it is now.
+ struct Peer: Equatable {
+ /// ICE `disconnected` or `failed`.
+ var connectionBad = false
+ /// No audio packets arrived at the last look, and they are not muted.
+ var audioMissing = false
+ }
+
+ private var streamDownSince: Double?
+ private var connectionBadSince: [String: Double] = [:]
+ private var audioMissingSince: [String: Double] = [:]
+ private var callClearSince: Double?
+ private var peerClearSince: [String: Double] = [:]
+
+ /// Whether the whole call is said to be reconnecting.
+ private(set) var call = false
+ /// The people whose tiles say "Reconnecting to …".
+ private(set) var peers: Set<String> = []
+
+ init() {}
+
+ /// One look. `streamDown` is the core's `reconnecting` phase; `people`
+ /// everyone in the call but you.
+ mutating func observe(at now: Double, streamDown: Bool, people: [String: Peer]) {
+ streamDownSince = streamDown ? (streamDownSince ?? now) : nil
+ for id in Array(connectionBadSince.keys) where people[id] == nil { connectionBadSince.removeValue(forKey: id) }
+ for id in Array(audioMissingSince.keys) where people[id] == nil { audioMissingSince.removeValue(forKey: id) }
+ var troubled: Set<String> = []
+ for (id, p) in people {
+ connectionBadSince[id] = p.connectionBad ? (connectionBadSince[id] ?? now) : nil
+ audioMissingSince[id] = p.audioMissing ? (audioMissingSince[id] ?? now) : nil
+ let connection = connectionBadSince[id].map { now - $0 >= Self.connectionAfter } ?? false
+ let audio = audioMissingSince[id].map { now - $0 >= Self.audioAfter } ?? false
+ if connection || audio { troubled.insert(id) }
+ }
+ let stream = streamDownSince.map { now - $0 >= Self.streamAfter } ?? false
+ let everyone = troubled.count >= 2 && troubled.count == people.count
+ let wantCall = stream || everyone
+ let wantPeers = wantCall ? [] : troubled
+
+ if wantCall {
+ call = true
+ callClearSince = nil
+ } else if call {
+ let since = callClearSince ?? now
+ callClearSince = since
+ if now - since >= Self.holdAfter {
+ call = false
+ callClearSince = nil
+ }
+ }
+
+ var shown = peers.filter { people[$0] != nil }
+ for id in wantPeers {
+ shown.insert(id)
+ peerClearSince.removeValue(forKey: id)
+ }
+ for id in shown where !wantPeers.contains(id) {
+ // Covered by the whole call's words, or better: held, then gone.
+ if wantCall {
+ shown.remove(id)
+ peerClearSince.removeValue(forKey: id)
+ continue
+ }
+ let since = peerClearSince[id] ?? now
+ peerClearSince[id] = since
+ if now - since >= Self.holdAfter {
+ shown.remove(id)
+ peerClearSince.removeValue(forKey: id)
+ }
+ }
+ for id in Array(peerClearSince.keys) where !shown.contains(id) { peerClearSince.removeValue(forKey: id) }
+ peers = shown
+ }
+}
+
+/// The words (spec §15.23).
+public enum ReconnectWords {
+ public static let call = "Reconnecting…"
+ public static func peer(_ name: String) -> String { "Reconnecting to \(name)\u{2026}" }
+}
+
+/// `log show --predicate 'subsystem == "net.amadan.oficina.calls" && category == "connection"' --info`,
+/// and the last lines in "Copy call diagnostics": when "Reconnecting…" was
+/// said and cleared.
+enum CallLog {
+ static let logger = Logger(subsystem: "net.amadan.oficina.calls", category: "connection")
+
+ static func info(_ message: String) {
+ logger.info("\(message, privacy: .public)")
+ AudioDiagnosticsStore.connection.log("info", message)
+ }
+}
+
+extension AudioDiagnosticsStore {
+ /// The connection log's lines, kept apart from the others.
+ public static let connection = AudioDiagnosticsStore()
+}
diff --git a/native/apple/Sources/CallsKit/ScreenWatchdog.swift b/native/apple/Sources/CallsKit/ScreenWatchdog.swift
index 8286145..dbcd5db 100644
--- a/native/apple/Sources/CallsKit/ScreenWatchdog.swift
+++ b/native/apple/Sources/CallsKit/ScreenWatchdog.swift
@@ -60,6 +60,29 @@ enum ScreenRecovery: Equatable, Sendable {
/// What stopping and sharing again did: this connection's screen
/// transceiver stopped and a new one added, renegotiated.
case reAdd(String)
+ /// A receiver's report that keyframes did not answer: this connection's
+ /// screen sending path built again — a new transceiver, so a new encoder
+ /// and, at the far end, a new receiver and decoder — at the same size
+ /// and mode. What stopping and sharing again does, for that one person.
+ case rebuild(String)
+ /// Still reported after that: every connection's screen stopped and
+ /// added again from the same capture (no picker), which is exactly what
+ /// Stop and Share did by hand.
+ case reshare
+}
+
+/// Where the answer to one receiver's report has got to (the sharer's
+/// notice says it in words: see ScreenWords.notice).
+enum ScreenEscalation: Equatable, Sendable {
+ case none
+ /// Keyframes asked for since this time.
+ case keyframes(since: Double)
+ /// That peer's screen rebuilt at this time.
+ case rebuilt(at: Double)
+ /// Everyone's screen shared again at this time.
+ case reshared(at: Double)
+ /// Nothing more to try: the person is told to stop and share again.
+ case failed
}
/// The sharer's watchdog: pure, fed once a second while sharing.
@@ -83,6 +106,17 @@ enum ScreenRecovery: Equatable, Sendable {
/// (the report is a few seconds old when it lands), three times and then
/// a rest — and the fuller steps only if the screen has stopped leaving
/// here too.
+/// - **A receiver keeps saying so** (spec §15.22, Paul 2026-09-30: "still
+/// seeing issues with screen sharing after a while", the notice sat at
+/// "recovering"). Keyframes cannot mend a decoder that has wedged at the
+/// far end, or a receiver that has stopped taking this stream at all;
+/// a new transceiver can. Still reported six seconds after the first
+/// keyframe: that connection's screen is rebuilt, at the same size and
+/// mode (a report alone never shrinks a Text and detail share). Still
+/// reported ten seconds after that: everyone's screen is shared again
+/// from the same capture. Ten seconds more and the sharer is told it
+/// could not be restored. A rebuild at most once a minute per person,
+/// a share again at most once every five minutes.
struct ScreenWatchdog: Equatable {
static let captureStallAfter = 3.0
static let captureBackoffFirst = 5.0
@@ -100,6 +134,14 @@ struct ScreenWatchdog: Equatable {
/// and that is not a stall (the motion gate, 2026-09-28). A receiver's
/// report is still answered then.
static let warmUp = 10.0
+ /// §15.22's escalation: how long a report may outlast the first
+ /// keyframe, then the rebuild, then the share again; and how often each
+ /// of the last two may happen.
+ static let rebuildAfter = 6.0
+ static let reshareAfter = 10.0
+ static let giveUpAfter = 10.0
+ static let rebuildEvery = 60.0
+ static let reshareEvery = 300.0
struct Peer: Equatable {
var last = ScreenSendCounts()
@@ -112,7 +154,17 @@ struct ScreenWatchdog: Equatable {
var reported = false
var restingUntil = 0.0
var since = 0.0
+ var escalation = ScreenEscalation.none
+ var lastRebuild: Double?
+ /// Frames sent on this connection since the escalation last moved on:
+ /// a receiver can only be failing to decode frames that were sent. A
+ /// screen the link is holding back, or one with nothing new to send,
+ /// is not taken further.
+ var sentSinceStage: UInt64 = 0
}
+ /// Frames that must leave for a receiver before its report is taken
+ /// further than keyframes.
+ static let escalateAfterSent: UInt64 = 3
private(set) var peers: [String: Peer] = [:]
private var heartbeats: UInt64?
@@ -128,6 +180,10 @@ struct ScreenWatchdog: Equatable {
private(set) var encoderRecoveries = 0
private(set) var reAdds = 0
private(set) var stallReports = 0
+ private(set) var rebuilds = 0
+ private(set) var reshares = 0
+ private(set) var failures = 0
+ private var lastReshare: Double?
/// One line on what the watchdog is doing, for the device panel.
private(set) var status = "watching"
@@ -177,6 +233,7 @@ struct ScreenWatchdog: Equatable {
// all, so the steps do not start again from nothing on the new one.
for (id, counts) in sending {
var peer = peers[id] ?? Peer(last: counts, movedAt: now, deliveredAt: delivered, since: now)
+ peer.sentSinceStage += counts.sent >= peer.last.sent ? counts.sent - peer.last.sent : counts.sent
if counts.encoded < peer.last.encoded || counts.sent < peer.last.sent {
// A new transceiver's counters, starting again from nothing.
peer.last = counts
@@ -238,20 +295,123 @@ struct ScreenWatchdog: Equatable {
}
peers[id] = peer
}
+ out += escalate(at: now, stalledBy: stalledBy, negotiated: Set(sending.keys))
status = describe()
return out
}
+ /// §15.22: a report that keyframes have not answered, taken further. Run
+ /// over every peer with a record, whether or not its screen is
+ /// negotiated this second — a rebuilt connection has none for a moment,
+ /// and a rebuild that never comes back must still end in words.
+ private mutating func escalate(
+ at now: Double, stalledBy: Set<String>, negotiated: Set<String>
+ ) -> [ScreenRecovery] {
+ var out: [ScreenRecovery] = []
+ var reshare = false
+ for id in peers.keys.sorted() {
+ guard var peer = peers[id] else { continue }
+ let reported = stalledBy.contains(id)
+ // Frames went to this receiver since the last step, or there is no
+ // screen negotiated with it at all (a rebuild that never came back).
+ let owed = peer.sentSinceStage >= Self.escalateAfterSent || !negotiated.contains(id)
+ let before = peer.escalation
+ defer {
+ if peers[id]?.escalation != before { peers[id]?.sentSinceStage = 0 }
+ }
+ if !reported {
+ if peer.escalation != .none {
+ switch peer.escalation {
+ case .keyframes: ScreenLog.info("\(Self.short(id))'s view restored after keyframes")
+ case .rebuilt: ScreenLog.info("\(Self.short(id))'s view restored after the rebuild")
+ case .reshared: ScreenLog.info("\(Self.short(id))'s view restored after sharing again")
+ case .failed: ScreenLog.info("\(Self.short(id))'s view came back after all")
+ case .none: break
+ }
+ peer.escalation = .none
+ }
+ peers[id] = peer
+ continue
+ }
+ let canRebuild = peer.lastRebuild.map { now - $0 >= Self.rebuildEvery } ?? true
+ let canReshare = reshare || (lastReshare.map { now - $0 >= Self.reshareEvery } ?? true)
+ switch peer.escalation {
+ case .none:
+ peer.escalation = .keyframes(since: now)
+ case .keyframes(let since) where now - since >= Self.rebuildAfter && owed:
+ if canRebuild {
+ out.append(.rebuild(id))
+ rebuilds += 1
+ peer.lastRebuild = now
+ peer.escalation = .rebuilt(at: now)
+ ScreenLog.info(
+ "\(Self.short(id)) still reports a stall \(Int(now - since)) s after keyframes: rebuilding its screen")
+ } else if canReshare {
+ reshare = true
+ peer.escalation = .reshared(at: now)
+ ScreenLog.info("\(Self.short(id)) still reports a stall, rebuilt under a minute ago: sharing again")
+ } else {
+ peer.escalation = .failed
+ failures += 1
+ ScreenLog.error("\(Self.short(id)) still reports a stall and every step is resting: giving up")
+ }
+ case .rebuilt(let at) where now - at >= Self.reshareAfter && owed:
+ if canReshare {
+ reshare = true
+ peer.escalation = .reshared(at: now)
+ ScreenLog.info("\(Self.short(id)) still reports a stall \(Int(now - at)) s after the rebuild: sharing again")
+ } else {
+ peer.escalation = .failed
+ failures += 1
+ ScreenLog.error("\(Self.short(id)) still reports a stall after the rebuild, shared again under five minutes ago: giving up")
+ }
+ case .reshared(let at) where now - at >= Self.giveUpAfter && owed:
+ peer.escalation = .failed
+ failures += 1
+ ScreenLog.error("\(Self.short(id)) still reports a stall \(Int(now - at)) s after sharing again: giving up")
+ default:
+ break
+ }
+ peers[id] = peer
+ }
+ if reshare {
+ out.append(.reshare)
+ reshares += 1
+ lastReshare = now
+ // One share again answers every report in flight.
+ for (id, peer) in peers where stalledBy.contains(id) {
+ switch peer.escalation {
+ case .keyframes, .rebuilt:
+ peers[id]?.escalation = .reshared(at: now)
+ peers[id]?.sentSinceStage = 0
+ default: break
+ }
+ }
+ }
+ return out
+ }
+
+ /// Where the answer to each reporting peer has got to, for the notice.
+ func escalation(of peer: String) -> ScreenEscalation { peers[peer]?.escalation ?? .none }
+
/// Whether anyone's view of the screen is being recovered right now.
var recovering: Bool { captureStalled || peers.values.contains { $0.step > 0 || $0.reported } }
private func describe() -> String {
if captureStalled { return "capture stalled, restarting (\(captureRestarts))" }
+ let stages = peers.values.map(\.escalation)
+ if stages.contains(.failed) { return "a receiver's view could not be restored" }
+ if stages.contains(where: { if case .reshared = $0 { return true } else { return false } }) {
+ return "shared again for everyone"
+ }
+ if stages.contains(where: { if case .rebuilt = $0 { return true } else { return false } }) {
+ return "rebuilt for a receiver"
+ }
if let (_, p) = peers.first(where: { $0.value.step > 0 }) {
return ["", "keyframe asked for", "encoder recreated", "re-added"][min(p.step, 3)]
}
if peers.values.contains(where: \.reported) { return "a receiver reports a stall" }
- let total = captureRestarts + keyframes + encoderRecoveries + reAdds
+ let total = captureRestarts + keyframes + encoderRecoveries + reAdds + rebuilds + reshares
return total == 0 ? "watching" : "watching, \(total) recover\(total == 1 ? "y" : "ies") so far"
}
@@ -261,21 +421,37 @@ struct ScreenWatchdog: Equatable {
/// The receiver's watchdog: whose screens have stopped moving here.
///
/// A peer's screen is stalled when nothing new has been decoded from it
-/// for four seconds while it says it is sharing. Both apps and the browser
-/// repeat a still screen at least once a second, so four seconds without
-/// a frame is not a still screen. The list is reported as `screenStalled`
-/// (spec §15.21) and the sharer recovers; it clears as soon as frames
-/// move, and a report is never held for more than a minute — a screen
-/// the sharer cannot recover is not made into a report that never ends.
+/// for four seconds while it says it is sharing. Every sender repeats a
+/// still screen at least once a second (the iPhone's broadcast too, since
+/// 2026-09-30), so four seconds without a frame is not a still screen. The
+/// list is reported as `screenStalled` (spec §15.21) and the sharer
+/// recovers; it clears as soon as frames move.
+///
+/// Any change in the count is movement, a smaller one included: a screen
+/// the sharer has rebuilt arrives on a new receiver whose count starts
+/// again from nothing (2026-09-30 — before this, a rebuilt screen that was
+/// moving again stayed "stalled" here until its new count passed the old
+/// one, and the sharer's notice with it).
+///
+/// Four seconds into a stall the tile's picture is also dropped and
+/// attached again, once (spec §15.22): a renderer that has stopped, on
+/// this side, is this side's to mend. A report is held for at most three
+/// minutes from the last frame, long enough for the sharer to try
+/// everything and say so, and no longer.
struct ScreenReceiveWatch: Equatable {
static let stallAfter = 4.0
- static let giveUpAfter = 60.0
+ static let reattachAfter = 4.0
+ static let giveUpAfter = 180.0
struct Seen: Equatable {
var decoded: UInt64
var movedAt: Double
+ var reattached = false
}
private(set) var seen: [String: Seen] = [:]
+ /// The peers whose picture should be attached again now: set by each
+ /// look, and each stall's only once.
+ private(set) var reattach: [String] = []
init() {}
@@ -284,18 +460,26 @@ struct ScreenReceiveWatch: Equatable {
mutating func observe(at now: Double, decoded: [String: UInt64]) -> [String] {
for id in seen.keys where decoded[id] == nil { seen.removeValue(forKey: id) }
var stalled: [String] = []
+ reattach = []
for (id, count) in decoded {
- guard var s = seen[id], count <= s.decoded else {
+ guard var s = seen[id], count == s.decoded else {
seen[id] = Seen(decoded: count, movedAt: now)
continue
}
- s.decoded = count
- seen[id] = s
let quiet = now - s.movedAt
if quiet >= Self.stallAfter && quiet < Self.giveUpAfter { stalled.append(id) }
+ if quiet >= Self.stallAfter + Self.reattachAfter && !s.reattached && quiet < Self.giveUpAfter {
+ s.reattached = true
+ reattach.append(id)
+ }
+ seen[id] = s
}
+ reattach.sort()
return stalled.sorted()
}
+
+ /// How long each stalled peer's screen has gone without a frame.
+ func quiet(at now: Double) -> [String: Double] { seen.mapValues { now - $0.movedAt } }
}
/// The words for the sharer's quiet notice.
@@ -310,4 +494,108 @@ public enum ScreenWords {
default: return "\(names.count) people\u{2019}s views of your screen stalled \u{2014} recovering"
}
}
+
+ /// The notice as the escalation moves on (spec §15.22): "recovering",
+ /// then "Restarting your share for Samuel…", then either nothing or
+ /// "Couldn’t restore Samuel’s view — try Stop and Share again". Worst
+ /// news first when several people are at different steps.
+ public static func notice(recovering: [String], restarting: [String], failed: [String]) -> String {
+ var parts: [String] = []
+ if !failed.isEmpty {
+ let whose: String
+ switch failed.count {
+ case 1: whose = "\(failed[0])\u{2019}s view"
+ case 2: whose = "\(failed[0])\u{2019}s and \(failed[1])\u{2019}s views"
+ default: whose = "\(failed.count) people\u{2019}s views"
+ }
+ parts.append("Couldn\u{2019}t restore \(whose) \u{2014} try Stop and Share again")
+ }
+ if !restarting.isEmpty {
+ let who: String
+ switch restarting.count {
+ case 1: who = restarting[0]
+ case 2: who = "\(restarting[0]) and \(restarting[1])"
+ default: who = "\(restarting.count) people"
+ }
+ parts.append("Restarting your share for \(who)\u{2026}")
+ }
+ let stalled = Self.recovering(names: recovering)
+ if !stalled.isEmpty { parts.append(stalled) }
+ return parts.joined(separator: ". ")
+ }
+}
+
+/// One peer's screen as it arrives here, from the receiver's statistics
+/// (spec §15.22): enough to tell the ways a screen stops apart. Packets
+/// arriving and frames assembled but none decoded is a decoder that has
+/// wedged; nothing arriving at all is the sender, or the path; frames
+/// decoded and a picture that does not move is the renderer.
+struct ScreenReceiveReport: Equatable, Sendable {
+ var packetsReceived: UInt64 = 0
+ var packetsLost: Int64 = 0
+ var bytesReceived: UInt64 = 0
+ var framesReceived: UInt64 = 0
+ var framesDecoded: UInt64 = 0
+ var framesDropped: UInt64 = 0
+ var keyFramesDecoded: UInt64 = 0
+ var pliCount: UInt64 = 0
+ var firCount: UInt64 = 0
+ var nackCount: UInt64 = 0
+ var freezeCount: UInt64 = 0
+ var width = 0
+ var height = 0
+ var decoder = ""
+ /// When the last packet arrived, in the statistics' own milliseconds.
+ var lastPacketAt = 0.0
+ var mid = ""
+
+ init() {}
+
+ /// One inbound-rtp report's values added in (a screen has one).
+ mutating func add(_ v: [String: NSObject]) {
+ func n(_ key: String) -> UInt64 { (v[key] as? NSNumber)?.uint64Value ?? 0 }
+ packetsReceived += n("packetsReceived")
+ packetsLost += (v["packetsLost"] as? NSNumber)?.int64Value ?? 0
+ bytesReceived += n("bytesReceived")
+ framesReceived += n("framesReceived")
+ framesDecoded += n("framesDecoded")
+ framesDropped += n("framesDropped")
+ keyFramesDecoded += n("keyFramesDecoded")
+ pliCount += n("pliCount")
+ firCount += n("firCount")
+ nackCount += n("nackCount")
+ freezeCount += n("freezeCount")
+ width = max(width, Int(n("frameWidth")))
+ height = max(height, Int(n("frameHeight")))
+ if let name = v["decoderImplementation"] as? String, !name.isEmpty { decoder = name }
+ lastPacketAt = max(lastPacketAt, (v["lastPacketReceivedTimestamp"] as? NSNumber)?.doubleValue ?? 0)
+ }
+
+ /// What changed between two looks, in words: "packets +120, frames
+ /// received +30, decoded +0, dropped +0, keyframes +0, PLI +2".
+ func moved(since before: ScreenReceiveReport?) -> String {
+ guard let b = before, b.mid == mid else { return summary }
+ func d(_ a: UInt64, _ z: UInt64) -> String { a >= z ? "+\(a - z)" : "new \(a)" }
+ return "packets \(d(packetsReceived, b.packetsReceived)), frames received \(d(framesReceived, b.framesReceived)), "
+ + "decoded \(d(framesDecoded, b.framesDecoded)), dropped \(d(framesDropped, b.framesDropped)), "
+ + "keyframes \(d(keyFramesDecoded, b.keyFramesDecoded)), PLI \(d(pliCount, b.pliCount)), "
+ + "NACK \(d(nackCount, b.nackCount))"
+ }
+
+ /// The running totals, one line.
+ var summary: String {
+ "mid \(mid.isEmpty ? "-" : mid), \(width)x\(height), decoder \(decoder.isEmpty ? "unknown" : decoder); "
+ + "packets \(packetsReceived) (lost \(packetsLost)), frames received \(framesReceived), "
+ + "decoded \(framesDecoded), dropped \(framesDropped), keyframes \(keyFramesDecoded), "
+ + "PLI \(pliCount), FIR \(firCount), NACK \(nackCount), freezes \(freezeCount)"
+ }
+
+ /// The verdict the numbers point to, for a screen that has stopped.
+ func diagnosis(since before: ScreenReceiveReport?) -> String {
+ guard let b = before, b.mid == mid else { return "no earlier reading" }
+ if packetsReceived == b.packetsReceived { return "no packets arriving (the sender or the path)" }
+ if framesReceived == b.framesReceived { return "packets arriving, no frames assembled (loss, or waiting on a keyframe)" }
+ if framesDecoded == b.framesDecoded { return "frames arriving, none decoded (the decoder)" }
+ return "decoding"
+ }
}
diff --git a/native/apple/Sources/CallsMac/Resources/MacIcon.png b/native/apple/Sources/CallsMac/Resources/MacIcon.png
index f0de939..1ba703e 100644
Binary files a/native/apple/Sources/CallsMac/Resources/MacIcon.png and b/native/apple/Sources/CallsMac/Resources/MacIcon.png differ
diff --git a/native/apple/Tests/CallsKitTests/ReconnectWatchTests.swift b/native/apple/Tests/CallsKitTests/ReconnectWatchTests.swift
new file mode 100644
index 0000000..8d656ff
--- /dev/null
+++ b/native/apple/Tests/CallsKitTests/ReconnectWatchTests.swift
@@ -0,0 +1,102 @@
+import XCTest
+
+@testable import CallsKit
+
+/// "Reconnecting…" only for a problem that lasts (spec §15.23). Paul,
+/// 2026-09-30: "I saw 'Reconnecting' appear occasionally over the video,
+/// but only for a second … I didn't notice any interruption."
+final class ReconnectWatchTests: XCTestCase {
+ private typealias Peer = ReconnectWatch.Peer
+ private let fine = Peer()
+
+ /// Runs the watch at 5 Hz from `from` to `to` seconds, and returns the
+ /// times at which the whole call's words were showing.
+ private func run(
+ _ watch: inout ReconnectWatch, from: Double, to: Double,
+ stream: (Double) -> Bool, people: (Double) -> [String: Peer]
+ ) -> (call: [Double], peers: [Double: Set<String>]) {
+ var call: [Double] = []
+ var peers: [Double: Set<String>] = [:]
+ var t = from
+ while t < to {
+ watch.observe(at: t, streamDown: stream(t), people: people(t))
+ if watch.call { call.append(t) }
+ if !watch.peers.isEmpty { peers[t] = watch.peers }
+ t = (t * 10 + 2).rounded() / 10
+ }
+ return (call, peers)
+ }
+
+ func testARoutineStreamRenewalIsNeverShown() {
+ // The stream ends on purpose and is back half a second later, over and
+ // over, and a slower resume of three and a half seconds as well.
+ var watch = ReconnectWatch()
+ let seen = run(
+ &watch, from: 0, to: 60,
+ stream: { t in (10..<10.6).contains(t) || (30..<33.5).contains(t) || (50..<50.4).contains(t) },
+ people: { _ in ["sam": self.fine] })
+ XCTAssertEqual(seen.call, [])
+ XCTAssertTrue(seen.peers.isEmpty)
+ }
+
+ func testAStreamDownFourSecondsIsShownAndHeldASecondPastRecovery() {
+ var watch = ReconnectWatch()
+ let seen = run(&watch, from: 0, to: 20, stream: { (5..<12).contains($0) }, people: { _ in ["sam": self.fine] })
+ XCTAssertEqual(seen.call.first, 9, "down at 5, said at 9")
+ XCTAssertEqual(seen.call.last, 12.8, "back at 12, said until a second after")
+ }
+
+ func testShortBlipsOnOnePersonAreNeverShown() {
+ var watch = ReconnectWatch()
+ let seen = run(
+ &watch, from: 0, to: 30, stream: { _ in false },
+ people: { t in
+ ["sam": Peer(connectionBad: (5..<7.5).contains(t), audioMissing: (15..<16.5).contains(t))]
+ })
+ XCTAssertEqual(seen.call, [])
+ XCTAssertTrue(seen.peers.isEmpty, "\(seen.peers)")
+ }
+
+ func testOnePersonsLastingTroubleIsSaidOnTheirTileAlone() {
+ var watch = ReconnectWatch()
+ let seen = run(
+ &watch, from: 0, to: 30, stream: { _ in false },
+ people: { t in ["sam": Peer(connectionBad: (5..<15).contains(t)), "ana": self.fine] })
+ XCTAssertEqual(seen.call, [], "not the whole call")
+ let times = seen.peers.keys.sorted()
+ XCTAssertEqual(times.first, 8, "disconnected at 5, said at 8")
+ XCTAssertEqual(times.last, 15.8, "held a second past recovery")
+ XCTAssertTrue(seen.peers.values.allSatisfy { $0 == ["sam"] })
+ }
+
+ func testAudioThatStopsFromSomeoneUnmutedIsSaidAfterTwoSeconds() {
+ var watch = ReconnectWatch()
+ let seen = run(
+ &watch, from: 0, to: 20, stream: { _ in false },
+ people: { t in ["sam": Peer(audioMissing: (5..<10).contains(t)), "ana": self.fine] })
+ XCTAssertEqual(seen.peers.keys.sorted().first, 7)
+ }
+
+ func testEveryonesTroubleIsTheWholeCall() {
+ var watch = ReconnectWatch()
+ let seen = run(
+ &watch, from: 0, to: 20, stream: { _ in false },
+ people: { t in
+ let bad = (5..<12).contains(t)
+ return ["sam": Peer(connectionBad: bad), "ana": Peer(connectionBad: bad)]
+ })
+ XCTAssertEqual(seen.call.first, 8)
+ XCTAssertTrue(seen.peers.isEmpty, "not two tiles as well")
+ }
+
+ func testTheWords() {
+ XCTAssertEqual(ReconnectWords.call, "Reconnecting…")
+ XCTAssertEqual(ReconnectWords.peer("Samuel"), "Reconnecting to Samuel…")
+ }
+
+ func testTheNoticeFollowsTheWatchNotThePhase() {
+ let notices = CallNotices.notices(
+ phase: "admitted", status: "In the meeting", sharingScreen: false, screenNote: "", floor: false)
+ XCTAssertFalse(notices.contains { $0.kind == .reconnecting })
+ }
+}
diff --git a/native/apple/Tests/CallsKitTests/ScreenWatchdogTests.swift b/native/apple/Tests/CallsKitTests/ScreenWatchdogTests.swift
index 3e125e3..85aa5c4 100644
--- a/native/apple/Tests/CallsKitTests/ScreenWatchdogTests.swift
+++ b/native/apple/Tests/CallsKitTests/ScreenWatchdogTests.swift
@@ -83,6 +83,25 @@ final class ScreenWatchdogTests: XCTestCase {
XCTAssertEqual(dog.reAdds, 1)
}
+ func testAnEncoderStoppedForOnePersonIsMendedForThemAlone() {
+ // Each connection has an encoder of its own: the browser's (H.264 at
+ // the level Chromium answers) can refuse frames the app's (High 5.2)
+ // takes. The steps go to the stopped one only.
+ var dog = ScreenWatchdog()
+ var log: [(Int, ScreenRecovery)] = []
+ for t in 0..<20 {
+ let delivered = UInt64(t * 30)
+ let sending = [
+ "browser": ScreenSendCounts(encoded: t < 5 ? delivered : 120, sent: t < 5 ? delivered : 120, keyframes: 1),
+ "app": ScreenSendCounts(encoded: delivered, sent: delivered, keyframes: 1),
+ ]
+ for a in dog.observe(at: Double(t), capture: counts(delivered, delivered), sending: sending, stalledBy: []) {
+ log.append((t, a))
+ }
+ }
+ XCTAssertEqual(log.map(\.1), [.keyframe("browser"), .recreateEncoder("browser"), .reAdd("browser")])
+ }
+
func testAnEncoderThatRecoversAfterTheKeyframeIsLeftAlone() {
var dog = ScreenWatchdog()
var actions: [ScreenRecovery] = []
@@ -126,7 +145,7 @@ final class ScreenWatchdogTests: XCTestCase {
}
}
- func testAReceiversReportWhileTheScreenLeavesNormallyGetsKeyframesOnly() {
+ func testAReceiversReportWhileTheScreenLeavesNormallyGetsKeyframesFirstAndNeverAShrink() {
var dog = ScreenWatchdog()
var log: [(Int, ScreenRecovery)] = []
for t in 0..<20 {
@@ -136,8 +155,12 @@ final class ScreenWatchdogTests: XCTestCase {
log.append((t, a))
}
}
- XCTAssertEqual(log.map(\.1), [.keyframe("vicky"), .keyframe("vicky"), .keyframe("vicky")])
- XCTAssertEqual(log.map(\.0), [3, 7, 11])
+ let keyframes = log.filter { $0.1 == .keyframe("vicky") }
+ XCTAssertEqual(keyframes.map(\.0), [3, 7, 11])
+ // What comes after the keyframes (spec §15.22) is a rebuild at the
+ // same size, never a smaller encoder.
+ XCTAssertFalse(log.contains { $0.1 == .recreateEncoder("vicky") })
+ XCTAssertEqual(log.first?.1, .keyframe("vicky"))
XCTAssertEqual(dog.stallReports, 1, "one report, however long it lasts")
XCTAssertTrue(dog.recovering)
}
@@ -182,17 +205,197 @@ final class ScreenWatchdogTests: XCTestCase {
}
}
+ // MARK: - A receiver that keeps reporting (spec §15.22)
+
+ private func escalating(
+ _ range: Range<Int>, stalledBy: (Int) -> Set<String>, sent: (Int) -> [String: ScreenSendCounts]? = { _ in nil },
+ dog: inout ScreenWatchdog
+ ) -> [(Int, ScreenRecovery)] {
+ var log: [(Int, ScreenRecovery)] = []
+ for t in range {
+ let n = UInt64(t * 30)
+ for a in dog.observe(at: Double(t), capture: counts(n, n), sending: sent(t) ?? sending(n), stalledBy: stalledBy(t)) {
+ log.append((t, a))
+ }
+ }
+ return log
+ }
+
+ func testAReportKeyframesDoNotAnswerIsRebuiltThenSharedAgainThenGivenUp() {
+ var dog = ScreenWatchdog()
+ let log = escalating(0..<45, stalledBy: { $0 >= 3 ? ["vicky"] : [] }, dog: &dog)
+ let steps = log.filter { if case .keyframe = $0.1 { return false } else { return true } }
+ // Reported from 3: keyframes at once; six seconds on, still reported,
+ // the connection is rebuilt; ten more, everyone's screen shared again;
+ // ten more, given up.
+ XCTAssertEqual(steps.map(\.1), [.rebuild("vicky"), .reshare])
+ XCTAssertEqual(steps.map(\.0), [9, 19])
+ XCTAssertEqual(dog.escalation(of: "vicky"), .failed)
+ XCTAssertEqual(dog.rebuilds, 1)
+ XCTAssertEqual(dog.reshares, 1)
+ XCTAssertEqual(dog.failures, 1)
+ // A report alone never shrinks the screen or recreates its encoder:
+ // Text and detail stays Text and detail.
+ XCTAssertFalse(log.contains { if case .recreateEncoder = $0.1 { return true } else { return false } })
+ XCTAssertEqual(dog.status, "a receiver's view could not be restored")
+ }
+
+ func testAFailureStaysUntilTheReportClears() {
+ var dog = ScreenWatchdog()
+ _ = escalating(0..<45, stalledBy: { $0 >= 3 ? ["vicky"] : [] }, dog: &dog)
+ let later = escalating(45..<120, stalledBy: { $0 < 100 ? ["vicky"] : [] }, dog: &dog)
+ XCTAssertFalse(later.contains { $0.1 == .reshare || $0.1 == .rebuild("vicky") }, "nothing more is tried")
+ XCTAssertEqual(dog.escalation(of: "vicky"), .none, "and a view that comes back is forgotten")
+ }
+
+ func testARebuildThatWorksEndsIt() {
+ var dog = ScreenWatchdog()
+ let log = escalating(0..<60, stalledBy: { (3..<12).contains($0) ? ["vicky"] : [] }, dog: &dog)
+ let steps = log.filter { if case .keyframe = $0.1 { return false } else { return true } }
+ XCTAssertEqual(steps.map(\.1), [.rebuild("vicky")])
+ XCTAssertEqual(dog.escalation(of: "vicky"), .none)
+ XCTAssertEqual(dog.reshares, 0)
+ }
+
+ func testRebuildsAndSharesAgainAreRationed() {
+ var dog = ScreenWatchdog()
+ // First stall: rebuilt at 9, and that works.
+ _ = escalating(0..<20, stalledBy: { (3..<12).contains($0) ? ["vicky"] : [] }, dog: &dog)
+ // A second, within the minute: no rebuild so soon, so straight to a
+ // share again; that works too.
+ let second = escalating(20..<50, stalledBy: { (25..<34).contains($0) ? ["vicky"] : [] }, dog: &dog)
+ XCTAssertEqual(second.filter { $0.1 == .reshare || $0.1 == .rebuild("vicky") }.map(\.1), [.reshare])
+ // A third, past the minute but inside five: rebuilt, and then there is
+ // no share again to be had, so it is given up.
+ let third = escalating(80..<120, stalledBy: { $0 >= 85 ? ["vicky"] : [] }, dog: &dog)
+ XCTAssertEqual(third.filter { $0.1 == .reshare || $0.1 == .rebuild("vicky") }.map(\.1), [.rebuild("vicky")])
+ XCTAssertEqual(dog.escalation(of: "vicky"), .failed)
+ }
+
+ func testNothingSentIsNothingToEscalate() {
+ // A still screen with nothing new to send, or a link holding it all
+ // back: the receiver cannot be failing to decode frames that never
+ // left, so keyframes are all it gets.
+ var dog = ScreenWatchdog()
+ var log: [(Int, ScreenRecovery)] = []
+ for t in 0..<40 {
+ // Reporting all along (idle reports), nothing new delivered or sent
+ // after t=3.
+ let n = UInt64(min(t, 3) * 30)
+ let still = ["vicky": ScreenSendCounts(encoded: n, sent: n, keyframes: 1)]
+ for a in dog.observe(
+ at: Double(t), capture: counts(UInt64(t * 30), n), sending: still, stalledBy: t >= 3 ? ["vicky"] : [])
+ {
+ log.append((t, a))
+ }
+ }
+ XCTAssertFalse(log.isEmpty)
+ XCTAssertTrue(log.allSatisfy { if case .keyframe = $0.1 { return true } else { return false } }, "\(log)")
+ XCTAssertEqual(dog.escalation(of: "vicky"), .keyframes(since: 3))
+ }
+
+ func testARebuildThatNeverComesBackStillEndsInWords() {
+ // The rebuilt connection never negotiates a screen again: there is
+ // nothing to count, and the steps go on regardless.
+ var dog = ScreenWatchdog()
+ let log = escalating(
+ 0..<45, stalledBy: { $0 >= 3 ? ["vicky"] : [] }, sent: { $0 > 9 ? [:] : nil }, dog: &dog)
+ XCTAssertTrue(log.contains { $0.1 == .reshare })
+ XCTAssertEqual(dog.escalation(of: "vicky"), .failed)
+ }
+
+ func testOneShareAgainAnswersEveryReport() {
+ var dog = ScreenWatchdog()
+ var log: [(Int, ScreenRecovery)] = []
+ for t in 0..<30 {
+ let n = UInt64(t * 30)
+ let both = ["vicky": ScreenSendCounts(encoded: n, sent: n, keyframes: 1), "tony": ScreenSendCounts(encoded: n, sent: n, keyframes: 1)]
+ let stalled: Set<String> = t >= 3 ? ["vicky", "tony"] : []
+ for a in dog.observe(at: Double(t), capture: counts(n, n), sending: both, stalledBy: stalled) { log.append((t, a)) }
+ }
+ XCTAssertEqual(log.filter { $0.1 == .reshare }.count, 1)
+ XCTAssertEqual(log.filter { if case .rebuild = $0.1 { return true } else { return false } }.count, 2)
+ }
+
+ func testTheNoticeMovesOnInWords() {
+ XCTAssertEqual(ScreenWords.notice(recovering: [], restarting: [], failed: []), "")
+ XCTAssertEqual(
+ ScreenWords.notice(recovering: ["Samuel"], restarting: [], failed: []),
+ "Samuel\u{2019}s view of your screen stalled \u{2014} recovering")
+ XCTAssertEqual(
+ ScreenWords.notice(recovering: [], restarting: ["Samuel"], failed: []), "Restarting your share for Samuel\u{2026}")
+ XCTAssertEqual(
+ ScreenWords.notice(recovering: [], restarting: [], failed: ["Samuel"]),
+ "Couldn\u{2019}t restore Samuel\u{2019}s view \u{2014} try Stop and Share again")
+ XCTAssertEqual(
+ ScreenWords.notice(recovering: ["Tony"], restarting: [], failed: ["Samuel"]),
+ "Couldn\u{2019}t restore Samuel\u{2019}s view \u{2014} try Stop and Share again. Tony\u{2019}s view of your screen stalled \u{2014} recovering")
+ XCTAssertEqual(
+ ScreenWords.notice(recovering: [], restarting: ["A", "B", "C"], failed: []), "Restarting your share for 3 people\u{2026}")
+ }
+
+ func testTheReceiversNumbersNameTheStall() {
+ var before = ScreenReceiveReport()
+ before.mid = "3"
+ before.packetsReceived = 1000
+ before.framesReceived = 300
+ before.framesDecoded = 300
+ var wedged = before
+ wedged.packetsReceived = 1400
+ wedged.framesReceived = 420
+ XCTAssertEqual(wedged.diagnosis(since: before), "frames arriving, none decoded (the decoder)")
+ XCTAssertTrue(wedged.moved(since: before).hasPrefix("packets +400, frames received +120, decoded +0"))
+ XCTAssertEqual(before.diagnosis(since: before), "no packets arriving (the sender or the path)")
+ var rebuilt = wedged
+ rebuilt.mid = "5"
+ XCTAssertEqual(rebuilt.diagnosis(since: before), "no earlier reading")
+ }
+
// MARK: - The receiver
- func testAScreenThatStopsMovingIsReportedAfterFourSecondsAndForAMinuteAtMost() {
+ func testAScreenThatStopsMovingIsReportedAfterFourSecondsAndForThreeMinutesAtMost() {
var watch = ScreenReceiveWatch()
var reported: [Int] = []
- for t in 0..<80 {
+ for t in 0..<200 {
let decoded: UInt64 = t < 10 ? UInt64(t * 10) : 90
if !watch.observe(at: Double(t), decoded: ["paul": decoded]).isEmpty { reported.append(t) }
}
XCTAssertEqual(reported.first, 13, "last moved at 9: stalled from 13")
- XCTAssertEqual(reported.last, 68, "and no longer than a minute after it last moved")
+ XCTAssertEqual(reported.last, 188, "and no longer than three minutes after it last moved")
+ }
+
+ func testARebuiltScreenStartingFromZeroIsMovingNotStalled() {
+ // 2026-09-30: a screen the sharer rebuilt arrives on a new receiver
+ // whose count starts again. Up to 0.4.4 only a larger count was
+ // movement, so a rebuilt screen that was moving again stayed stalled
+ // here — and the sharer's notice with it — until the new count passed
+ // the old.
+ var watch = ScreenReceiveWatch()
+ for t in 0..<10 { _ = watch.observe(at: Double(t), decoded: ["paul": 900 + UInt64(t * 30)]) }
+ for t in 10..<16 { _ = watch.observe(at: Double(t), decoded: ["paul": 1170]) }
+ XCTAssertEqual(watch.observe(at: 16, decoded: ["paul": 1170]), ["paul"])
+ for t in 17..<40 {
+ XCTAssertEqual(watch.observe(at: Double(t), decoded: ["paul": UInt64((t - 17) * 30)]), [], "t=\(t)")
+ }
+ }
+
+ func testAStalledPictureIsAttachedAgainOnceFourSecondsIn() {
+ var watch = ScreenReceiveWatch()
+ var reattached: [Int] = []
+ for t in 0..<60 {
+ let decoded: UInt64 = t < 10 ? UInt64(t * 10) : 90
+ _ = watch.observe(at: Double(t), decoded: ["paul": decoded])
+ if watch.reattach == ["paul"] { reattached.append(t) }
+ }
+ XCTAssertEqual(reattached, [17], "stalled from 13, attached again four seconds later, once")
+ // Moving again and stalled again: a new stall, a new try.
+ for t in 60..<63 { _ = watch.observe(at: Double(t), decoded: ["paul": UInt64(t)]) }
+ var again: [Int] = []
+ for t in 63..<80 {
+ _ = watch.observe(at: Double(t), decoded: ["paul": 62])
+ if !watch.reattach.isEmpty { again.append(t) }
+ }
+ XCTAssertEqual(again, [70])
}
func testAMovingScreenIsNeverReported() {
diff --git a/native/ios/Sources/PhoneFullScreenCall.swift b/native/ios/Sources/PhoneFullScreenCall.swift
index 4676f9b..6640a90 100644
--- a/native/ios/Sources/PhoneFullScreenCall.swift
+++ b/native/ios/Sources/PhoneFullScreenCall.swift
@@ -266,7 +266,7 @@ extension PhoneCall {
private var notices: [CallNotice] {
CallNotices.notices(
- phase: call.state?.phase, status: call.status, sharingScreen: call.sharingScreen,
+ phase: call.noticePhase, status: call.status, sharingScreen: call.sharingScreen,
screenNote: call.screenNote, floor: call.floor, screenRecovery: call.screenRecovery)
}
diff --git a/native/ios/project.yml b/native/ios/project.yml
index c72fb95..295a52a 100644
--- a/native/ios/project.yml
+++ b/native/ios/project.yml
@@ -10,7 +10,7 @@ settings:
base:
SWIFT_VERSION: '5.0'
CODE_SIGN_STYLE: Automatic
- MARKETING_VERSION: '0.4.4'
+ MARKETING_VERSION: '0.4.5'
CURRENT_PROJECT_VERSION: '1'
# The App Group the app and its broadcast extension share: the socket
# the iPhone's screen crosses lives in its container. One setting
diff --git a/test/browser/call.mjs b/test/browser/call.mjs
index 204a71e..59058f8 100644
--- a/test/browser/call.mjs
+++ b/test/browser/call.mjs
@@ -2607,9 +2607,30 @@ const run = {
const before = { a: await state(pageA), b: await state(pageB) };
// Long enough for at least one renewal each at a six-second life,
- // and in practice two.
- await sleep(16000);
+ // and in practice two. Watched four times a second the whole way:
+ // a renewal is a stream ended on purpose and resumed, and §15.23 says
+ // nobody sees it — not "Reconnecting…" over the call, not on a tile,
+ // not in the title (Paul, 2026-09-30: "I saw 'Reconnecting' appear
+ // occasionally over the video, but only for a second").
+ const said = [];
+ let streamDowns = 0;
+ for (const until = Date.now() + 16000; Date.now() < until;) {
+ for (const [page, who] of [[pageA, 'A'], [pageB, 'B']]) {
+ const r = await page.evaluate(() => ({
+ reconnect: window.meet.state().reconnect,
+ status: (document.getElementById('status') || {}).textContent || '',
+ pill: (document.getElementById('call-state') || {}).textContent || '',
+ tiles: Array.from(document.querySelectorAll('.calls-tile-notice, [data-hear-notice]')).map((n) => n.textContent),
+ }));
+ if (r.reconnect.streamDown) streamDowns += 1;
+ const text = [r.status, r.pill, ...r.tiles].join(' | ');
+ if (r.reconnect.call || r.reconnect.peers.length || /Reconnecting/.test(text)) said.push(who + ': ' + text);
+ }
+ await sleep(250);
+ }
const after = { a: await state(pageA), b: await state(pageB) };
+ check(said.length === 0, 'no renewal was said as "Reconnecting" (' + said.slice(0, 3).join('; ') + ')');
+ console.log('renewal: a stream was seen down (between its end and its resume) at ' + streamDowns + ' of the looks');
check(after.a.peer === before.a.peer,
'A kept its participation across the renewals (' + before.a.peer + ' -> ' + after.a.peer + ')');
diff --git a/test/browser/native-screen.mjs b/test/browser/native-screen.mjs
index 637b9d0..c4551d3 100644
--- a/test/browser/native-screen.mjs
+++ b/test/browser/native-screen.mjs
@@ -21,6 +21,24 @@
// sender deactivated under it): the watchdog must recover it
// loss the browser stops painting and asks for a keyframe
// (screenStalled): the app must answer with one
+// wedge the browser's screen receiver stops passing anything to its
+// decoder (an encoded transform that drops every frame), which
+// no keyframe mends: the app must escalate to a rebuild and
+// the browser decode again within 30 s, the app's notice going
+// "recovering", "Restarting your share for …", then nothing
+// (spec §15.22)
+// wedge-sticky every screen receiver the browser gets is wedged, new
+// ones too: the app must end at "Couldn’t restore …", and
+// clear once the wedge is lifted
+// soak renewals and resumes as `renewals`, for SCREEN_SOAK_SECONDS,
+// with the mode cycled (balanced, motion, detail) and the
+// app's uplink capped and lifted every SCREEN_SOAK_STEP_MS:
+// every freeze is logged with what the ladder did about it
+// web-soak the same with the browser sharing and the app watching, the
+// browser's mode cycled
+// web-wedge, web-wedge-sticky the same the other way round: the
+// browser shares, the app's screen decoder wedges, and the
+// browser's ladder has to restore it (or say it cannot)
//
// Run by TestNativeMacScreenRecovery (make native-browser).
import { chromium } from 'playwright';
@@ -41,23 +59,104 @@ const browser = await chromium.launch({ headless: true, args: [
'--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream',
'--autoplay-policy=no-user-gesture-required', '--disable-features=WebRtcHideLocalIpsWithMdns',
'--mute-audio',
+ // getDisplayMedia answered with a fake 1280x720 monitor and no picker,
+ // for the web-* scenarios (see call.mjs).
+ '--auto-accept-this-tab-capture',
] });
-const context = await browser.newContext({ permissions: ['camera', 'microphone'] });
+// bypassCSP: the wedge's encoded transform runs in a worker made from a
+// blob, which the page's own policy (default-src 'self') would refuse.
+// The gate's, never the page's.
+const context = await browser.newContext({ permissions: ['camera', 'microphone'], bypassCSP: true });
await context.addCookies([{ name: process.env.CALLS_COOKIE_NAME, value: process.env.CALLS_COOKIE_A, url: origin }]);
const page = await context.newPage();
await page.addInitScript(() => {
const Native = window.RTCPeerConnection;
window.__nativeInteropPCs = new Set();
+ // §15.22's wedge: a receiver whose frames never reach its decoder.
+ // Chromium takes an encoded transform only on a receiver that has not
+ // started, so in the wedge scenarios every screen receiver gets one as
+ // it arrives — passing everything through — and __wedgeScreens tells
+ // the ones there now to swallow every frame instead. `sticky` has each
+ // one that arrives after start out swallowing.
+ let worker = null;
+ let serial = 0;
+ const wedgeWorker = () => {
+ if (!worker) {
+ const code = [
+ 'const drops = new Map();',
+ 'onmessage = (m) => { for (const id of m.data.ids) drops.set(id, m.data.drop); };',
+ 'onrtctransform = (e) => {',
+ ' const id = e.transformer.options.id;',
+ ' if (!drops.has(id)) drops.set(id, !!e.transformer.options.drop);',
+ ' e.transformer.readable.pipeThrough(new TransformStream({ transform(f, c) { if (!drops.get(id)) c.enqueue(f); } }))',
+ ' .pipeTo(e.transformer.writable);',
+ '};',
+ ].join('\n');
+ worker = new Worker(URL.createObjectURL(new Blob([code], { type: 'text/javascript' })));
+ }
+ return worker;
+ };
+ window.__wedge = { sticky: false, wedged: 0, ids: new Map() };
+ const screenReceiver = (tx) => tx && tx.receiver && tx.receiver.track && tx.receiver.track.kind === 'video' &&
+ tx.direction === 'recvonly' && tx.currentDirection !== 'stopped';
+ const watchReceiver = (tx) => {
+ if (!window.__wedgeReady || !screenReceiver(tx) || window.__wedge.ids.has(tx.receiver)) return;
+ const id = ++serial;
+ window.__wedge.ids.set(tx.receiver, id);
+ tx.receiver.transform = new RTCRtpScriptTransform(wedgeWorker(), { id, drop: window.__wedge.sticky });
+ if (window.__wedge.sticky) window.__wedge.wedged += 1;
+ };
+ window.__wedgeScreens = (sticky) => {
+ window.__wedge.sticky = !!sticky;
+ const ids = [];
+ for (const pc of window.__nativeInteropPCs) {
+ for (const tx of pc.getTransceivers()) {
+ const id = screenReceiver(tx) && window.__wedge.ids.get(tx.receiver);
+ if (id) ids.push(id);
+ }
+ }
+ wedgeWorker().postMessage({ ids, drop: true });
+ window.__wedge.wedged += ids.length;
+ return ids.length;
+ };
+ window.__releaseWedge = () => {
+ window.__wedge.sticky = false;
+ wedgeWorker().postMessage({ ids: [...window.__wedge.ids.values()], drop: false });
+ };
window.RTCPeerConnection = class extends Native {
- constructor(...args) { super(...args); window.__nativeInteropPCs.add(this); }
+ // The web-* scenarios' screen goes out as H.264: the app can only
+ // wedge a VideoToolbox decoder (DecoderWedge.swift), and the browser
+ // left to itself sends VP8. What is under test, the ladder, does not
+ // turn on the codec.
+ addTransceiver(track, init) {
+ const tx = super.addTransceiver(track, init);
+ if (window.__h264Screen && init && init.direction === 'sendonly' && track && track.kind === 'video') {
+ const codecs = RTCRtpSender.getCapabilities('video').codecs;
+ const h264 = codecs.filter((c) => /h264/i.test(c.mimeType));
+ try {
+ tx.setCodecPreferences([...h264, ...codecs.filter((c) => !/h264/i.test(c.mimeType))]);
+ } catch {}
+ }
+ return tx;
+ }
+ constructor(...args) {
+ super(...args);
+ window.__nativeInteropPCs.add(this);
+ this.addEventListener('track', (e) => watchReceiver(e.transceiver));
+ }
};
});
+// Receivers made ready to wedge only in the scenarios that wedge them, so
+// every other scenario's browser decodes exactly as a person's would.
+await page.addInitScript((ready) => { window.__wedgeReady = ready; }, scenario === 'wedge' || scenario === 'wedge-sticky');
let native;
let failed = false;
try {
await page.goto(`${origin}/m/${room}`);
await page.waitForFunction(() => !!window.meet);
- native = spawn(binary, [], { stdio: ['pipe', 'pipe', 'pipe'] });
+ // The app's decoders can be wedged in the web-* scenarios only.
+ const env = scenario.startsWith('web-') ? Object.assign({}, process.env, { CALLS_DECODER_WEDGE: '1' }) : process.env;
+ native = spawn(binary, [], { stdio: ['pipe', 'pipe', 'pipe'], env });
native.stdin.write(process.env.CALLS_NATIVE_INPUT + '\n');
let sample = null;
let exited = false;
@@ -82,6 +181,17 @@ try {
};
await wait(() => sample?.stats.framesDecoded > 15, 'native receives the browser');
const nativePeer = sample.peer;
+ if (scenario === 'web-soak') {
+ await webSoak(page, native, () => sample, wait);
+ console.log(`PASS ${scenario}`);
+ throw new Done();
+ }
+ if (scenario.startsWith('web-')) {
+ await webShares(page, native, () => sample, replies, wait, scenario.endsWith('sticky'));
+ console.log(`PASS ${scenario}`);
+ process.exitCode = 0;
+ throw new Done();
+ }
const key = nativePeer + '#share';
// What the browser is receiving on the app's screen, from the
// connection's own statistics: the screen is the inbound video that is
@@ -110,7 +220,12 @@ try {
// The mode first: it is remembered across runs of the interop binary,
// and the screen's source is made for the mode the share starts in.
native.stdin.write(`screen-mode ${process.env.SCREEN_MODE || 'balanced'}\n`);
+ // SCREEN_SOURCE=motion shares the 1920-wide scrolling page (its height
+ // CALLS_SYNTHETIC_SCREEN_HEIGHT) rather than the small still screen:
+ // what meets H.264 level 3.1's limit (§15.22).
if (scenario === 'display') native.stdin.write(`screen-display ${process.env.SCREEN_DEPTH || 6}\n`);
+ else if (process.env.SCREEN_SOURCE === 'motion') native.stdin.write('screen-motion\n');
+ else if (process.env.SCREEN_SOURCE === 'display') native.stdin.write(`screen-display ${process.env.SCREEN_DEPTH || 6}\n`);
else native.stdin.write('screen-on\n');
await wait(() => sample?.sharingScreen === true, 'native says it is sharing');
await wait(async () => (await received()).frames > 15, 'the browser decodes the app’s screen');
@@ -172,7 +287,22 @@ try {
await watchFor(SOAK, 'display');
clearInterval(tick);
} else if (scenario === 'renewals' || scenario === 'resume') {
+ // §15.23: a renewal or a resume is never said as "Reconnecting…", by
+ // the app or the browser, while the core's phase does go through it.
+ const said = [];
+ let phases = 0;
+ const saidTick = setInterval(async () => {
+ if (sample?.phase === 'reconnecting') phases += 1;
+ if (sample?.reconnecting || sample?.reconnectingPeers) said.push('app');
+ try {
+ const r = (await page.evaluate(() => window.meet.state())).reconnect;
+ if (r.call || r.peers.length) said.push('browser');
+ } catch {}
+ }, 250);
await watchFor(SOAK, scenario);
+ clearInterval(saidTick);
+ console.log(`reconnecting: the app's core was mid-resume at ${phases} samples; said by ${JSON.stringify([...new Set(said)])}`);
+ if (asserting) assert.equal(said.length, 0, 'a renewal or resume was said as Reconnecting: ' + said.join(','));
console.log(`ICE restarts seen by the browser: ${restarts}`);
const ours = sample?.stats.iceRestarts || 0;
console.log(`ICE restarts the app started: ${ours}`);
@@ -199,6 +329,76 @@ try {
await wait(() => (sample?.stats.screenWatchdogKeyframes || 0) + (sample?.stats.screenWatchdogEncoderRecoveries || 0) > 0,
'the watchdog says it recovered the encoder', 5);
console.log('PASS a stalled encoder was recovered without a re-share');
+ } else if (scenario === 'soak') {
+ // §15.22's search for the cause: a long share with everything that
+ // moves under it moving. What matters is not whether a freeze
+ // happened but what it took to end it — the ladder's counts at each
+ // freeze say whether keyframes were enough.
+ const modes = ['balanced', 'motion', 'detail'];
+ const uplinks = [0, 1500000, 600000, 0, 3000000, 900000];
+ const started = Date.now();
+ const notices = new Set();
+ let step = 0;
+ const tick = setInterval(() => {
+ step += 1;
+ const mode = modes[step % modes.length];
+ const up = uplinks[step % uplinks.length];
+ native.stdin.write(`screen-mode ${mode}\n`);
+ native.stdin.write(`uplink ${up}\n`);
+ const st = sample?.stats || {};
+ console.log(`soak ${Math.round((Date.now() - started) / 1000)} s: mode ${mode}, uplink ${up || 'open'} (was sending ${st.screenSendWidth}x${st.screenSendHeight} at ${(st.screenSendFpsTenths || 0) / 10} fps, limitation ${st.screenLimitation})`);
+ }, Number(process.env.SCREEN_SOAK_STEP_MS || 45000));
+ const noticeTick = setInterval(() => { if (sample?.screenRecovery) notices.add(sample.screenRecovery); }, 500);
+ await watchFor(SOAK, 'soak');
+ clearInterval(tick);
+ clearInterval(noticeTick);
+ native.stdin.write('uplink 0\n');
+ const st = sample?.stats || {};
+ console.log(`SOAK app: ICE restarts ${st.iceRestarts}, stall reports ${st.screenStallReports}, keyframes ${st.screenWatchdogKeyframes}, fresh encoders ${st.screenWatchdogEncoderRecoveries}, re-adds ${st.screenWatchdogReAdds}, rebuilds ${st.screenWatchdogRebuilds}, shares again ${st.screenWatchdogReshares}, given up ${st.screenWatchdogFailures}, capture restarts ${st.screenWatchdogCaptureRestarts}`);
+ console.log(`SOAK browser ICE restarts ${restarts}; notices at the app: ${JSON.stringify([...notices])}`);
+ console.log('the browser\u2019s screen log:\n' + (await page.evaluate(() => window.meet.diagnostics())).split('## Screen')[1]);
+ if (asserting) {
+ assert.equal(st.screenWatchdogFailures || 0, 0, 'the app gave up on the browser\u2019s view');
+ assert.equal(freezes.filter((f) => f.open).length, 0, 'the screen froze and never came back');
+ }
+ } else if (scenario === 'wedge' || scenario === 'wedge-sticky') {
+ await watchFor(5, 'before the wedge');
+ const sticky = scenario === 'wedge-sticky';
+ const keyframes = sample?.stats.screenWatchdogKeyframes || 0;
+ const notices = new Set();
+ const note = () => { if (sample?.screenRecovery) notices.add(sample.screenRecovery); };
+ const wedgedAt = Date.now();
+ const n = await page.evaluate((s) => window.__wedgeScreens(s), sticky);
+ assert.ok(n >= 1, 'the browser\u2019s screen receiver was wedged');
+ await wait(async () => (await page.evaluate(() => window.meet.state().screenStalled)).length === 1,
+ 'the browser reports the app\u2019s screen stalled', 15);
+ if (!sticky) {
+ // Painted frames on the share tile, counted across a new tile.
+ const progress = await progressOf(() => received().then((r) => r.frames));
+ await wait(async () => { note(); return (await progress()) > 15; },
+ 'the browser decodes the app\u2019s screen again, without a share again', 30);
+ console.log(`the browser decoding again ${Date.now() - wedgedAt} ms after the wedge: rebuilds ${sample?.stats.screenWatchdogRebuilds}, shares again ${sample?.stats.screenWatchdogReshares}, keyframes ${(sample?.stats.screenWatchdogKeyframes || 0) - keyframes}`);
+ assert.ok((sample?.stats.screenWatchdogKeyframes || 0) > keyframes, 'keyframes were tried first');
+ assert.ok((sample?.stats.screenWatchdogRebuilds || 0) > 0, 'the rebuild is what mended it');
+ await wait(async () => (await page.evaluate(() => window.meet.state().screenStalled)).length === 0,
+ 'the browser clears its report', 20);
+ await wait(() => { note(); return sample?.screenRecovery === ''; }, 'the app\u2019s notice clears', 20);
+ console.log('notices seen at the app: ' + JSON.stringify([...notices]));
+ assert.ok([...notices].some((x) => x.includes('recovering')), 'the notice said recovering');
+ assert.ok([...notices].some((x) => x.includes('Restarting your share for')), 'the notice said restarting');
+ console.log('PASS a wedged browser receiver was restored by the app\u2019s rebuild');
+ } else {
+ await wait(() => { note(); return (sample?.screenRecovery || '').includes('Couldn\u2019t restore'); },
+ 'the app says it could not restore the browser\u2019s view', 60);
+ console.log('notices seen at the app: ' + JSON.stringify([...notices]));
+ assert.ok((sample?.stats.screenWatchdogRebuilds || 0) > 0 && (sample?.stats.screenWatchdogReshares || 0) > 0,
+ 'a rebuild and a share again were tried first');
+ assert.ok([...notices].some((x) => x.includes('Restarting your share for')), 'the notice said restarting');
+ await page.evaluate(() => window.__releaseWedge());
+ await recovered('the browser decodes again once the wedge is lifted');
+ await wait(() => sample?.screenRecovery === '', 'the app\u2019s notice clears once the view is back', 30);
+ console.log('PASS the app said it could not restore a view nothing could, and cleared when it came back');
+ }
} else if (scenario === 'loss') {
await watchFor(5, 'before loss');
// The browser's watch treats the app's screen as frozen, as it would
@@ -231,8 +431,10 @@ try {
}
console.log(`PASS ${scenario}: the app’s screen kept arriving`);
} catch (err) {
- failed = true;
- console.log('FAIL', err.message);
+ if (!(err instanceof Done)) {
+ failed = true;
+ console.log('FAIL', err.message);
+ }
} finally {
if (native) {
native.stdin.write('leave\n');
@@ -242,3 +444,128 @@ try {
await browser.close();
if (failed) process.exit(1);
}
+
+// Done ends a scenario that has passed before the app-shares path below
+// it: thrown, and caught as not a failure.
+function Done() {}
+
+// progressOf counts how far a counter has moved since now, across a
+// restart from zero (a new tile, a new receiver): a fall is taken as a
+// new count starting.
+async function progressOf(read) {
+ let last = await read();
+ let total = 0;
+ return async () => {
+ const now = await read();
+ total += now >= last ? now - last : now;
+ last = now;
+ return total;
+ };
+}
+
+// webShares is the web-* scenarios: the browser shares its screen (a fake
+// monitor), the app watches, the app's screen decoder is wedged, and the
+// browser's own ladder (§15.22) has to put it right — or, sticky, say it
+// cannot, and clear once the wedge is lifted.
+async function webShares(page, native, current, replies, wait, sticky) {
+ const stat = (k) => current()?.stats[k] || 0;
+ await page.evaluate(() => { window.__h264Screen = true; });
+ await page.hover('#share');
+ await page.click('#share');
+ await wait(() => stat('remoteScreens') === 1 && stat('screenFramesDecoded') > 15, 'the app decodes the browser\u2019s screen', 30);
+ console.log('PASS the app decodes the browser\u2019s screen');
+ await new Promise((r) => setTimeout(r, 5000));
+ const s0 = () => page.evaluate(() => window.meet.state());
+ const keyframes = (await s0()).shareRecoveries.keyframes;
+ const wedgedAt = Date.now();
+ native.stdin.write(sticky ? 'screen-decoder-wedge sticky\n' : 'screen-decoder-wedge\n');
+ await wait(() => replies.some((r) => (r.decoderWedged || 0) >= 1), 'the app\u2019s screen decoder is wedged', 10);
+ await wait(() => stat('screenStalledHere') === 1, 'the app reports the browser\u2019s screen stalled (the wedge hit the screen)', 15);
+ const notices = new Set();
+ const note = async () => {
+ const s = await s0();
+ if (s.shareRecovery) notices.add(s.shareRecovery);
+ return s;
+ };
+ if (!sticky) {
+ const base = stat('screenFramesDecodedTotal');
+ await wait(async () => { await note(); return stat('screenFramesDecodedTotal') > base + 15; },
+ 'the app decodes the browser\u2019s screen again, without a share again', 30);
+ const s = await note();
+ console.log(`the app decoding again ${Date.now() - wedgedAt} ms after the wedge: ${JSON.stringify(s.shareRecoveries)}`);
+ assert.ok(s.shareRecoveries.keyframes > keyframes, 'keyframes were tried first');
+ assert.ok(s.shareRecoveries.rebuilds > 0, 'the rebuild is what mended it');
+ await wait(() => stat('screenStalledHere') === 0, 'the app clears its report', 20);
+ await wait(async () => (await note()).shareRecovery === '', 'the browser\u2019s notice clears', 20);
+ console.log('notices seen at the browser: ' + JSON.stringify([...notices]));
+ assert.ok([...notices].some((x) => x.includes('recovering')), 'the notice said recovering');
+ assert.ok([...notices].some((x) => x.includes('Restarting your share for')), 'the notice said restarting');
+ } else {
+ await wait(async () => (await note()).shareRecovery.includes('Couldn\u2019t restore'),
+ 'the browser says it could not restore the app\u2019s view', 60);
+ const s = await note();
+ console.log('notices seen at the browser: ' + JSON.stringify([...notices]) + ' ' + JSON.stringify(s.shareRecoveries));
+ assert.ok(s.shareRecoveries.rebuilds > 0 && s.shareRecoveries.reshares > 0, 'a rebuild and a share again were tried first');
+ assert.ok([...notices].some((x) => x.includes('Restarting your share for')), 'the notice said restarting');
+ native.stdin.write('screen-decoder-release\n');
+ const base = stat('screenFramesDecodedTotal');
+ await wait(() => stat('screenFramesDecodedTotal') > base + 10, 'the app decodes again once the wedge is lifted', 30);
+ await wait(async () => (await note()).shareRecovery === '', 'the browser\u2019s notice clears once the view is back', 30);
+ }
+ console.log('the browser\u2019s diagnostics:\n' + (await page.evaluate(() => window.meet.diagnostics())).split('## Screen')[1]);
+}
+
+// webSoak is `soak` with the browser sharing and the app watching: the
+// app's received screen, counted across rebuilds, watched for freezes
+// while the renewals run and the browser's mode is cycled.
+async function webSoak(page, native, current, wait) {
+ const stat = (k) => current()?.stats[k] || 0;
+ await page.hover('#share');
+ await page.click('#share');
+ await wait(() => stat('remoteScreens') === 1 && stat('screenFramesDecoded') > 15, 'the app decodes the browser\u2019s screen', 30);
+ const modes = ['balanced', 'motion', 'detail'];
+ const started = Date.now();
+ let step = 0;
+ const notices = new Set();
+ const freezes = [];
+ let last = stat('screenFramesDecodedTotal');
+ let moved = Date.now();
+ let frozen = null;
+ let nextStep = Date.now() + Number(process.env.SCREEN_SOAK_STEP_MS || 45000);
+ while (Date.now() - started < SOAK * 1000) {
+ await new Promise((r) => setTimeout(r, 1000));
+ const s = await page.evaluate(() => window.meet.state());
+ if (s.shareRecovery) notices.add(s.shareRecovery);
+ if (Date.now() >= nextStep) {
+ step += 1;
+ const mode = modes[step % modes.length];
+ await page.evaluate((m) => {
+ const el = document.getElementById('pick-share-mode');
+ if (el) { el.value = m; el.dispatchEvent(new Event('change', { bubbles: true })); }
+ }, mode);
+ console.log(`web-soak ${Math.round((Date.now() - started) / 1000)} s: the browser\u2019s mode ${mode}`);
+ nextStep = Date.now() + Number(process.env.SCREEN_SOAK_STEP_MS || 45000);
+ }
+ const now = stat('screenFramesDecodedTotal');
+ if (now > last) {
+ if (frozen) {
+ frozen.ms = Date.now() - frozen.since;
+ freezes.push(frozen);
+ console.log(`FREEZE web-soak: ${frozen.ms} ms, then frames again; browser ladder ${JSON.stringify(s.shareRecoveries)}`);
+ frozen = null;
+ }
+ moved = Date.now();
+ } else if (!frozen && Date.now() - moved > FREEZE_MS) {
+ frozen = { since: moved };
+ console.log(`FROZEN web-soak: no screen frame at the app for ${Date.now() - moved} ms; app stalled-here ${stat('screenStalledHere')}; browser ladder ${JSON.stringify(s.shareRecoveries)}`);
+ }
+ last = now;
+ }
+ const s = await page.evaluate(() => window.meet.state());
+ console.log(`SUMMARY web-soak: ${freezes.length} freezes, longest ${Math.max(0, ...freezes.map((f) => f.ms))} ms, open ${!!frozen}; app ICE restarts ${stat('iceRestarts')}, pictures attached again ${stat('screenReattaches')}; browser ladder ${JSON.stringify(s.shareRecoveries)}; notices ${JSON.stringify([...notices])}`);
+ console.log('the browser\u2019s screen log:\n' + (await page.evaluate(() => window.meet.diagnostics())).split('## Screen')[1]);
+ if (asserting) {
+ assert.equal(s.shareRecoveries.failures, 0, 'the browser gave up on the app\u2019s view');
+ assert.ok(!frozen, 'the app\u2019s screen froze and never came back');
+ }
+}
diff --git a/test/browser/room-reconnect.test.mjs b/test/browser/room-reconnect.test.mjs
new file mode 100644
index 0000000..e5a3455
--- /dev/null
+++ b/test/browser/room-reconnect.test.mjs
@@ -0,0 +1,105 @@
+// room-reconnect.test.mjs — "Reconnecting…" only for a problem that
+// lasts (spec §15.23, 2026-09-30), gated without a browser.
+//
+// node --test test/browser/room-reconnect.test.mjs
+//
+// reconnectStep and reconnectRecord lifted out of the shipped room.js by
+// name, with the constants they read, as room-screen.test.mjs does.
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+
+const src = readFileSync(fileURLToPath(new URL('../../internal/calls/static/room.js', import.meta.url)), 'utf8');
+
+function lift(name) {
+ const start = src.indexOf('function ' + name + '(');
+ assert.notEqual(start, -1, name + ' is not in room.js any more — this test is stale, not passing');
+ let depth = 0;
+ for (let i = src.indexOf('{', start); i < src.length; i++) {
+ if (src[i] === '{') depth += 1;
+ else if (src[i] === '}') {
+ depth -= 1;
+ if (depth === 0) return src.slice(start, i + 1);
+ }
+ }
+ assert.fail('unbalanced braces reading ' + name);
+}
+function liftConst(name) {
+ const m = src.match(new RegExp('const ' + name + ' = ([0-9.]+);'));
+ assert.ok(m, name + ' is not a numeric const in room.js any more');
+ return 'const ' + name + ' = ' + m[1] + ';';
+}
+const CONSTS = ['RECONNECT_STREAM_AFTER', 'RECONNECT_CONNECTION_AFTER', 'RECONNECT_AUDIO_AFTER', 'RECONNECT_HOLD']
+ .map(liftConst).join('\n');
+// eslint-disable-next-line no-new-func
+const { reconnectRecord, reconnectStep } = new Function(CONSTS + '\n' + lift('reconnectRecord') + '\n' +
+ lift('reconnectStep') + '; return { reconnectRecord, reconnectStep };')();
+
+const fine = { connectionBad: false, audioMissing: false };
+
+// Five looks a second from `from` to `to` seconds; the times the whole
+// call's words showed, and the tiles' at each time.
+function run(rec, from, to, stream, people) {
+ const call = [];
+ const peers = new Map();
+ for (let t = from * 10; t < to * 10; t += 2) {
+ const s = t / 10;
+ reconnectStep(rec, s * 1000, stream(s), people(s));
+ if (rec.call) call.push(s);
+ if (rec.peers.size) peers.set(s, Array.from(rec.peers).sort().join());
+ }
+ return { call, peers };
+}
+
+test('a routine stream renewal, and a slow one, are never said', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 60, (t) => (t >= 10 && t < 10.6) || (t >= 30 && t < 33.5),
+ () => new Map([['sam', fine]]));
+ assert.deepEqual(seen.call, []);
+ assert.equal(seen.peers.size, 0);
+});
+
+test('a stream down four seconds is said, and held a second past recovery', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 20, (t) => t >= 5 && t < 12, () => new Map([['sam', fine]]));
+ assert.equal(seen.call[0], 9);
+ assert.equal(seen.call[seen.call.length - 1], 12.8);
+});
+
+test('short blips on one person are never said', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 30, () => false, (t) => new Map([['sam', {
+ connectionBad: t >= 5 && t < 7.5, audioMissing: t >= 15 && t < 16.5,
+ }]]));
+ assert.deepEqual(seen.call, []);
+ assert.equal(seen.peers.size, 0);
+});
+
+test('one person\'s lasting trouble is said on their tile alone', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 30, () => false,
+ (t) => new Map([['sam', { connectionBad: t >= 5 && t < 15, audioMissing: false }], ['ana', fine]]));
+ assert.deepEqual(seen.call, []);
+ const times = Array.from(seen.peers.keys());
+ assert.equal(times[0], 8);
+ assert.equal(times[times.length - 1], 15.8);
+ assert.ok(Array.from(seen.peers.values()).every((v) => v === 'sam'));
+});
+
+test('audio that stops from someone unmuted is said after two seconds', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 20, () => false,
+ (t) => new Map([['sam', { connectionBad: false, audioMissing: t >= 5 && t < 10 }], ['ana', fine]]));
+ assert.equal(Array.from(seen.peers.keys())[0], 7);
+});
+
+test('everyone\'s trouble is the whole call, not a tile each', () => {
+ const rec = reconnectRecord();
+ const seen = run(rec, 0, 20, () => false, (t) => {
+ const bad = { connectionBad: t >= 5 && t < 12, audioMissing: false };
+ return new Map([['sam', bad], ['ana', bad]]);
+ });
+ assert.equal(seen.call[0], 8);
+ assert.equal(seen.peers.size, 0);
+});
diff --git a/test/browser/room-screen.test.mjs b/test/browser/room-screen.test.mjs
index 652228d..1aa5403 100644
--- a/test/browser/room-screen.test.mjs
+++ b/test/browser/room-screen.test.mjs
@@ -7,8 +7,12 @@
// does, with the constants they read.
//
// judgeScreen the receiver's verdict on one screen: stalled after
-// four seconds without a new frame, for a minute at most
-// recoveringWords the sharer's quiet notice
+// four seconds without a new frame, for three minutes at
+// most; any change in the count is movement
+// reattachDue the receiver's own mending, once per stall (§15.22)
+// escalateShare the sharer's ladder when keyframes do not answer
+// recoveringWords, noticeWords the sharer's notice
+// readingMoved what a stalled screen's statistics say
// mediaFrom, mediaKey, mediaPayload the report's screenStalled list
import { test } from 'node:test';
import assert from 'node:assert/strict';
@@ -37,13 +41,21 @@ function liftConst(name) {
assert.ok(m, name + ' is not a numeric const in room.js any more');
return 'const ' + name + ' = ' + m[1] + ';';
}
-const CONSTS = ['SCREEN_STALL_AFTER', 'SCREEN_STALL_GIVE_UP'].map(liftConst).join('\n');
+const CONSTS = [
+ 'SCREEN_STALL_AFTER', 'SCREEN_STALL_GIVE_UP', 'SCREEN_REATTACH_AFTER', 'SCREEN_REBUILD_AFTER',
+ 'SCREEN_RESHARE_AFTER', 'SCREEN_GIVE_UP_AFTER', 'SCREEN_REBUILD_EVERY',
+].map(liftConst).join('\n');
const build = (name) =>
// eslint-disable-next-line no-new-func
new Function(CONSTS + '\n' + lift(src, name) + '; return ' + name + ';')();
const judgeScreen = build('judgeScreen');
const recoveringWords = build('recoveringWords');
+const reattachDue = build('reattachDue');
+const escalateShare = build('escalateShare');
+const readingMoved = new Function(lift(src, 'readingSummary') + '\n' + lift(src, 'readingMoved') + '; return readingMoved;')();
+// eslint-disable-next-line no-new-func
+const noticeWords = new Function(lift(src, 'recoveringWords') + '\n' + lift(src, 'noticeWords') + '; return noticeWords;')();
const mediaFrom = build('mediaFrom');
const mediaPayload = build('mediaPayload');
// eslint-disable-next-line no-new-func
@@ -51,15 +63,108 @@ const mediaKey = new Function(lift(src, 'mediaKey') + '; return mediaKey;')();
const ID = '0123456789abcdef0123456789abcdef';
-test('a screen is stalled four seconds after its last frame, and for a minute at most', () => {
+test('a screen is stalled four seconds after its last frame, and for three minutes at most', () => {
const seen = { decoded: 0, movedAt: 0 };
const stalledAt = [];
- for (let t = 0; t <= 80; t++) {
+ for (let t = 0; t <= 200; t++) {
const decoded = t < 10 ? t * 10 : 90;
if (judgeScreen(seen, decoded, t * 1000, false)) stalledAt.push(t);
}
assert.equal(stalledAt[0], 13);
- assert.equal(stalledAt[stalledAt.length - 1], 68);
+ assert.equal(stalledAt[stalledAt.length - 1], 188);
+});
+
+test('a rebuilt screen whose count starts again is moving, not stalled', () => {
+ const seen = { decoded: 0, movedAt: 0 };
+ for (let t = 0; t < 10; t++) judgeScreen(seen, 900 + t * 30, t * 1000, false);
+ for (let t = 10; t < 16; t++) judgeScreen(seen, 1170, t * 1000, false);
+ assert.equal(judgeScreen(seen, 1170, 16000, false), true);
+ for (let t = 17; t < 40; t++) {
+ assert.equal(judgeScreen(seen, (t - 17) * 30, t * 1000, false), false, 't=' + t);
+ }
+});
+
+test('a stalled picture is handed to its tile again once, four seconds into the stall', () => {
+ const seen = { decoded: 0, movedAt: 0 };
+ const due = [];
+ for (let t = 0; t < 60; t++) {
+ const decoded = t < 10 ? t * 10 : 90;
+ judgeScreen(seen, decoded, t * 1000, false);
+ if (reattachDue(seen, t * 1000)) due.push(t);
+ }
+ assert.deepEqual(due, [17], 'last moved at 9, stalled from 13, handed back at 17, once');
+ judgeScreen(seen, 91, 60000, false);
+ const again = [];
+ for (let t = 61; t < 80; t++) {
+ judgeScreen(seen, 91, t * 1000, false);
+ if (reattachDue(seen, t * 1000)) again.push(t);
+ }
+ assert.deepEqual(again, [68], 'a new stall is a new try');
+});
+
+// The ladder, driven once a second the way answerScreenStalls drives it.
+function ladder(from, to, reported, opts = {}) {
+ const rec = opts.rec || { stage: '', at: 0, lastRebuild: 0 };
+ let resharedAt = opts.resharedAt || 0;
+ const out = [];
+ for (let t = from; t < to; t++) {
+ const now = t * 1000;
+ if (!reported(t)) {
+ rec.stage = '';
+ continue;
+ }
+ const owed = opts.owed ? opts.owed(t) : true;
+ const free = !resharedAt || now - resharedAt >= 300000;
+ const action = escalateShare(rec, now, owed, free);
+ if (action === 'reshare') resharedAt = now;
+ if (action) out.push([t, action]);
+ }
+ return { out, rec, resharedAt };
+}
+
+test('a report keyframes do not answer is rebuilt, then shared again, then given up', () => {
+ const { out, rec } = ladder(0, 45, (t) => t >= 3);
+ assert.deepEqual(out, [[9, 'rebuild'], [19, 'reshare'], [29, 'failed']]);
+ assert.equal(rec.stage, 'failed');
+});
+
+test('a rebuild that works ends the ladder', () => {
+ const { out, rec } = ladder(0, 60, (t) => t >= 3 && t < 12);
+ assert.deepEqual(out, [[9, 'rebuild']]);
+ assert.equal(rec.stage, '');
+});
+
+test('rebuilds and shares again are rationed', () => {
+ const first = ladder(0, 20, (t) => t >= 3 && t < 12);
+ const second = ladder(20, 50, (t) => t >= 25 && t < 34, { rec: first.rec, resharedAt: first.resharedAt });
+ assert.deepEqual(second.out, [[31, 'reshare']], 'no second rebuild inside a minute');
+ const third = ladder(80, 120, (t) => t >= 85, { rec: second.rec, resharedAt: second.resharedAt });
+ assert.deepEqual(third.out, [[91, 'rebuild'], [101, 'failed']], 'and no second share again inside five');
+});
+
+test('nothing sent is nothing to escalate', () => {
+ const { out, rec } = ladder(0, 60, (t) => t >= 3, { owed: () => false });
+ assert.deepEqual(out, []);
+ assert.equal(rec.stage, 'keyframes');
+});
+
+test('the notice moves on in words, the app\'s own', () => {
+ assert.equal(noticeWords([], [], []), '');
+ assert.equal(noticeWords(['Samuel'], [], []), 'Samuel’s view of your screen stalled — recovering');
+ assert.equal(noticeWords([], ['Samuel'], []), 'Restarting your share for Samuel…');
+ assert.equal(noticeWords([], [], ['Samuel']), 'Couldn’t restore Samuel’s view — try Stop and Share again');
+ assert.equal(noticeWords(['Tony'], [], ['Samuel']),
+ 'Couldn’t restore Samuel’s view — try Stop and Share again. Tony’s view of your screen stalled — recovering');
+ assert.equal(noticeWords([], ['A', 'B', 'C'], []), 'Restarting your share for 3 people…');
+});
+
+test('a stalled screen\'s statistics name what stopped', () => {
+ const base = { mid: '3', packets: 1000, lost: 0, received: 300, decoded: 300, dropped: 0, keyframes: 2, pli: 0, fir: 0,
+ nack: 0, freezes: 0, width: 1920, height: 1080, decoder: 'ExternalDecoder', muted: false };
+ const wedged = Object.assign({}, base, { packets: 1400, received: 420, dropped: 110 });
+ assert.match(readingMoved(wedged, base), /^packets \+400, frames received \+120, decoded \+0, dropped \+110.* — frames arriving, none decoded \(the decoder\)$/);
+ assert.match(readingMoved(base, base), /no packets arriving/);
+ assert.match(readingMoved(Object.assign({}, base, { mid: '5' }), base), /no earlier reading$/);
});
test('a moving screen is never stalled, and a frozen test window is', () => {
diff --git a/test/browser/room-share.test.mjs b/test/browser/room-share.test.mjs
index 7be3622..b12fd74 100644
--- a/test/browser/room-share.test.mjs
+++ b/test/browser/room-share.test.mjs
@@ -1001,11 +1001,14 @@ test('the screen is read off its own outbound report, by mid', () => {
const reports = [
{ mid: '1', frameWidth: 640, frameHeight: 480, framesPerSecond: 30 },
{ mid: '2', frameWidth: 1920, frameHeight: 1080, framesPerSecond: 8, qualityLimitationReason: 'bandwidth',
- targetBitrate: 1200000, encoderImplementation: 'OpenH264', framesEncoded: 400 },
+ targetBitrate: 1200000, encoderImplementation: 'OpenH264', framesEncoded: 400, framesSent: 398,
+ keyFramesEncoded: 3 },
];
+ // framesSent and keyFramesEncoded (§15.22): what the ladder counts as
+ // having left for this person, and the diagnostics' keyframes.
assert.deepEqual(shareSendReport(reports, '2'), {
width: 1920, height: 1080, fps: 8, limitation: 'bandwidth', targetBitrate: 1200000,
- encoder: 'OpenH264', framesEncoded: 400,
+ encoder: 'OpenH264', framesEncoded: 400, framesSent: 398, keyFramesEncoded: 3,
});
assert.equal(shareSendReport(reports, ''), null, 'not sharing on this connection');
assert.equal(shareSendReport(reports, '9'), null, 'not negotiated yet');