amadan
Say who can't hear you yet on the Mac and iPhone, and write §15.24
The apps report hearing (HearingRule: connected with packets in and decoded frames at the tap, or connected and muted), and ConnectingWatch says "<name> is connecting and can't hear you yet" on the tile, with dots that take turns, past a one-second grace until their report lists you, replaced by their blocked or no-sound report, with the fallback for clients that do not say and a twenty-second cap. Reconnecting waits until after. The mini window shows a small ear for anyone it is not showing, and the tab a dot. Snapshots for Mac (CallsInterop --connecting-snapshots) and iPhone (testConnectingSnapshots); the native-browser gate holds the browser's signalling and checks the notice both ways.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
15 files changed,
+914
−25
docs/superpowers/specs/2026-09-07-meet-design.md+99 −0native/README.md+9 −0native/apple/Sources/CallsInterop/CallSnapshots.swift+46 −0native/apple/Sources/CallsInterop/Interop.swift+6 −0native/apple/Sources/CallsKit/AudioMeter.swift+63 −8native/apple/Sources/CallsKit/AudioWork.swift+6 −0native/apple/Sources/CallsKit/Call.swift+112 −8native/apple/Sources/CallsKit/CallFixtures.swift+8 −0native/apple/Sources/CallsKit/ConnectingWatch.swift+160 −0native/apple/Sources/CallsKit/Media.swift+51 −7native/apple/Sources/CallsKit/MiniWindow.swift+52 −1native/apple/Sources/CallsKit/Wire.swift+6 −1native/apple/Tests/CallsKitTests/ConnectingWatchTests.swift+231 −0native/ios/UITests/CallsPhoneUITests.swift+24 −0test/browser/native.mjs+41 −0
diff --git a/docs/superpowers/specs/2026-09-07-meet-design.md b/docs/superpowers/specs/2026-09-07-meet-design.md| index 243ec8e..97222f9 100644 |
| --- a/docs/superpowers/specs/2026-09-07-meet-design.md |
| +++ b/docs/superpowers/specs/2026-09-07-meet-design.md |
| @@ -2566,3 +2566,102 @@ the apps, `RECONNECT_*` in room.js) with unit tests either side; the browser |
| gate's `renewal` scenario and the native gate's `resume` and `renewals` watch |
| for the words through several renewals and fail on any. The core's phase is |
| unchanged: controls that need an admitted call still follow it. |
| + |
| +### 15.24 §15.19, §15.20 — "… is connecting and can't hear you yet" (2026-10-01) |
| + |
| +**Why.** Paul, 2026-10-01, asking for what Zoom shows: "didemeski is |
| +connecting to audio and cannot hear you yet …". When someone joins, there is |
| +a gap — a few hundred milliseconds on a good network, seconds on a bad one, |
| +for ever on a broken one — before their side is receiving and playing your |
| +audio, and nothing said so: people greeted a newcomer who could not yet hear |
| +them. Nothing on the wire said whether anyone could hear anyone; §15.20 only |
| +carried bad news after five seconds of flat sound. |
| + |
| +**Decision.** A positive receive signal, and a notice built on it. |
| + |
| +- **`hearing`.** §15.19's `media` gains one more optional member: the peers |
| + whose audio this client is actually receiving and playing. Unlike §15.20's |
| + and §15.21's lists it is good news, and its presence is itself news: an |
| + absent list is a client that does not say (one from before this section), |
| + `[]` is a client that hears nobody yet. So it is sent whenever the server |
| + takes it, `[]` included, and the hub keeps it with its emptiness — `[]` |
| + travels as `[]`, and a change between absent and `[]` is a change. |
| + Otherwise it is held as `noSoundFrom` is: at most fifteen distinct |
| + well-formed peer ids, checked for shape and not presence, a `null`, a |
| + non-array, a repeat or a sixteenth id refused 400; re-encoded last |
| + (`muted`, `cameraOff`, `audioBlocked`, `noSoundFrom`, `screenStalled`, |
| + `hearing`). The raw payload cap is 3 KB, room for all three lists full |
| + (about 1,700 bytes) and whitespace. Sent only on a change, like the rest. |
| +- **The rule — who a client lists.** A peer is listed once the audio path |
| + from them is established and is playing here: |
| + - *established*: the connection is up (ICE connected or completed) and |
| + either their audio is arriving — inbound packets received and decoded |
| + samples coming out (`totalSamplesReceived` in the browser; frames at the |
| + receive tap in the apps) on a live receiver — or **they say they are |
| + muted**, in which case the connection being up and the receiver live |
| + (or no audio receiver at all, for someone with no microphone) is enough. |
| + A muted sender may send only silence, or nothing, and must not look |
| + unheard: the moment they unmute, it plays. Once established it stays so |
| + until the connection fails or closes; a blip is not losing them. |
| + - *playing*: in the browser, the tile's element is playing and was not |
| + refused by an autoplay policy (a refused tile is §15.20's |
| + `audioBlocked`, and is not listed until a click plays it). The apps play |
| + whatever they decode; there is no policy to refuse them. |
| +- **The notice.** From the moment a person appears in the roster — or |
| + appears again, or is given a new connection (a new pair generation) — |
| + their tile says "Samuel is connecting and can't hear you yet", with three |
| + dots after it that brighten in turn (one steady ellipsis under reduced |
| + motion). It goes the moment their report lists you in `hearing`, and a |
| + screen reader is told "Samuel can hear you now". In order: |
| + - **The grace.** It is not said at all if they hear you within 1 s of |
| + appearing (`CONNECTING_GRACE`, `ConnectingWatch.grace`), so ordinary |
| + joins say nothing. The browser gate measured 284 ms from appearing to |
| + being heard on a loopback. |
| + - **More specific news replaces it**: their `audioBlocked` ("Samuel's |
| + browser is blocking sound — they need to click to hear you"), or you in |
| + their `noSoundFrom` while you are unmuted ("Samuel isn't getting any |
| + sound from you"). Those are their own reports and show even inside the |
| + grace. |
| + - **"Reconnecting to Samuel…" waits until after** (§15.23): a connection |
| + that has never been up is not reconnecting. While someone is in their |
| + first moments the reconnect watch is told nothing is wrong with them, |
| + so a slow join of two people is never "Reconnecting…" for the call. |
| + - **A client that does not say** (no `hearing` in its report — a client |
| + from before this section, or any client on a server that refused the |
| + key) is let go once the connection to it is up and either its receiver |
| + reports show your audio reaching it (`remote-inbound-rtp` for your |
| + microphone, which exists only once one has come back, with packets |
| + sent) or it has made a media report without the list. A client that |
| + does say is believed over this evidence. |
| + - **The cap.** Whatever happens, it is given up 20 s after they appeared |
| + (`CONNECTING_CAP`, `ConnectingWatch.cap`), silently; §15.20's "isn't |
| + getting any sound" and §15.23's "Reconnecting to …" speak for any |
| + problem after that. |
| + - **Your own mute does not matter**: it is about their being able to |
| + hear you once you speak. |
| +- **Where.** On the person's tile, in the place and the glass of the other |
| + tile notices, with an ear where they have a warning triangle: every |
| + layout on the Mac and the iPhone, and the browser's call window. In the |
| + browser and in the apps' Grid, Speaker and Classic every person in the |
| + roster has a tile from the roster on, so there is no call-level form; |
| + the Mac's mini window, which shows one person, puts a small ear on its |
| + glass for anyone else still connecting (their names in its tooltip and |
| + for VoiceOver), and its tab shows the same ear. Screen readers hear the |
| + sentence once when it appears and "… can hear you now" when it goes, |
| + politely, and never the dots. |
| + |
| +**Compatibility, both orders.** Old clients ignore the member (the browser's |
| +`mediaFrom`, the core's decoder and the apps' `PeerMedia` all drop unknown |
| +keys). A server from before this section refuses a report carrying it |
| +(400): the browser then drops the extras for the rest of the page (§15.20) |
| +— though the browser's page is served by the same server, so only a page |
| +open across a deploy meets that — and the apps' core resends the same |
| +report without `hearing` and leaves it out from then on, then without |
| +`screenStalled` too if that is refused next, so a mute is never lost. The |
| +people on new clients then fall back to their own evidence about that |
| +client, as they do for an old one. So the server and the clients may ship |
| +in either order. |
| + |
| +**Not logged, not stored.** As §15.19. The apps log "… can hear us" in the |
| +connection log (`os_log` category `connection`), the browser on its |
| +console, each with the first six characters of the peer id only. |
diff --git a/native/README.md b/native/README.md| index bfefb36..2dc0fb9 100644 |
| --- a/native/README.md |
| +++ b/native/README.md |
| @@ -135,6 +135,15 @@ because nothing on it measured sound. Now: |
| isn't getting any sound from you". Each line is in the tile's accessibility |
| label and announced when it appears. The app itself is never blocked from |
| playing sound, so it never reports `audioBlocked`. |
| +- **Who can hear you yet** (spec §15.24): the app reports `hearing`, the people |
| + whose audio path is established here (connected, packets in and decoded |
| + frames at the tap, or connected and muted), read five times a second until |
| + it is. From the moment someone joins, their tile says "Samuel is connecting |
| + and can't hear you yet" — past a one-second grace, with dots that take turns |
| + (steady under Reduce Motion) — until their report lists you, their own |
| + blocked or no-sound report replacing it, and never past twenty seconds |
| + (`ConnectingWatch.swift`). The mini window and its tab show a small ear for |
| + anyone it is not showing. |
| The rules (`AudioLevelRules.swift`) are held by `AudioLevelTests` on an |
| injected clock; `TestNativeMacCallsBrowser` checks that the browser measures |
diff --git a/native/apple/Sources/CallsInterop/CallSnapshots.swift b/native/apple/Sources/CallsInterop/CallSnapshots.swift| index adfe63b..192da92 100644 |
| --- a/native/apple/Sources/CallsInterop/CallSnapshots.swift |
| +++ b/native/apple/Sources/CallsInterop/CallSnapshots.swift |
| @@ -60,6 +60,52 @@ import SwiftUI |
| } |
| } |
| + /// Spec §15.24: two people who have just joined — Samuel still |
| + /// connecting and unable to hear you yet, Bea's browser blocking sound in |
| + /// place of that — in Grid, Speaker and Classic, light and dark, over a |
| + /// busy picture, and the mini window and its tab with the small ear for |
| + /// someone it is not showing. |
| + static func saveConnecting(to folder: URL) async throws { |
| + NSApplication.shared.setActivationPolicy(.prohibited) |
| + try FileManager.default.createDirectory(at: folder, withIntermediateDirectories: true) |
| + let size = CGSize(width: 1000, height: 640) |
| + for scheme in [ColorScheme.light, .dark] { |
| + let suffix = LobbySnapshots.name(scheme) |
| + func shot( |
| + _ name: String, grid: Bool = false, style: CallLayoutStyle = .fullWindow, |
| + backdrop: FixtureBackdrop? = nil |
| + ) async throws { |
| + let call = Call(playAudio: false) |
| + CallFixture.connecting.apply(to: call) |
| + try await LobbySnapshots.render( |
| + MacCallView( |
| + call: call, grid: grid, showingChat: false, showingDevices: false, style: style, |
| + controlsShown: true, leave: {}, rejoin: {} |
| + ).tint(CallsAppearance.accent).environment(\.fixtureBackdrop, backdrop), |
| + scheme: scheme, size: size, |
| + to: folder.appendingPathComponent("mac-connecting-\(name)-\(suffix).png")) |
| + } |
| + try await shot("grid", grid: true) |
| + try await shot("speaker") |
| + try await shot("classic-grid", grid: true, style: .classic) |
| + try await shot("busy-grid", grid: true, backdrop: .busy) |
| + for hovering in [false, true] { |
| + let call = Call(playAudio: false) |
| + CallFixture.connecting.apply(to: call) |
| + let width = MiniWindowRules.size.width |
| + try await LobbySnapshots.render( |
| + MiniWindowSnapshot.window(call: call, hovering: hovering), scheme: scheme, |
| + size: CGSize(width: width, height: width / MiniWindowRules.aspect), |
| + to: folder.appendingPathComponent("mac-connecting-mini\(hovering ? "-hover" : "")-\(suffix).png")) |
| + } |
| + let call = Call(playAudio: false) |
| + CallFixture.connecting.apply(to: call) |
| + try await LobbySnapshots.render( |
| + MiniWindowSnapshot.tab(call: call), scheme: scheme, size: MiniWindowTab.size, |
| + to: folder.appendingPathComponent("mac-connecting-tab-\(suffix).png")) |
| + } |
| + } |
| + |
| /// The glass and the glow as the window server draws them (0.3.7). An |
| /// off-screen bitmap has no Liquid Glass and no backdrop to blur, so |
| /// these are real windows, parked below the desktop where nobody sees |
diff --git a/native/apple/Sources/CallsInterop/Interop.swift b/native/apple/Sources/CallsInterop/Interop.swift| index 21ef045..af6e902 100644 |
| --- a/native/apple/Sources/CallsInterop/Interop.swift |
| +++ b/native/apple/Sources/CallsInterop/Interop.swift |
| @@ -34,6 +34,12 @@ struct Input: Decodable { |
| } catch { exit(1) } |
| return |
| } |
| + if CommandLine.arguments.count == 3, CommandLine.arguments[1] == "--connecting-snapshots" { |
| + do { |
| + try await CallSnapshots.saveConnecting(to: URL(fileURLWithPath: CommandLine.arguments[2])) |
| + } catch { exit(1) } |
| + return |
| + } |
| if CommandLine.arguments.count == 3, CommandLine.arguments[1] == "--glass-snapshots" { |
| do { |
| try await CallSnapshots.saveGlass(to: URL(fileURLWithPath: CommandLine.arguments[2])) |
diff --git a/native/apple/Sources/CallsKit/AudioMeter.swift b/native/apple/Sources/CallsKit/AudioMeter.swift| index ccf985d..a8312ba 100644 |
| --- a/native/apple/Sources/CallsKit/AudioMeter.swift |
| +++ b/native/apple/Sources/CallsKit/AudioMeter.swift |
| @@ -46,15 +46,19 @@ import SwiftUI |
| @Published public private(set) var speaking = false |
| /// Their report about you: blocked, or not hearing you. |
| @Published public private(set) var notices: [String] = [] |
| + /// "<name> is connecting and can't hear you yet" (spec §15.24), drawn |
| + /// with a moving ellipsis; never beside `notices`. |
| + @Published public private(set) var connecting: String? |
| /// The glow behind their name. |
| public let glow = GlowLevel() |
| public init() {} |
| - func show(state: PeerAudioState, speaking: Bool, notices: [String]) { |
| + func show(state: PeerAudioState, speaking: Bool, notices: [String], connecting: String? = nil) { |
| if self.state != state { self.state = state } |
| if self.speaking != speaking { self.speaking = speaking } |
| if self.notices != notices { self.notices = notices } |
| + if self.connecting != connecting { self.connecting = connecting } |
| } |
| } |
| @@ -124,7 +128,8 @@ import SwiftUI |
| /// Returns the state the rule arrived at. |
| @discardableResult |
| func updatePeer( |
| - _ id: String, name: String, sample: PeerAudioSample, notices: [String], now: Double |
| + _ id: String, name: String, sample: PeerAudioSample, notices: [String], connecting: String? = nil, |
| + now: Double |
| ) -> PeerAudioState { |
| var watch = watches[id] ?? PeerAudioWatch() |
| let before = watch.state |
| @@ -132,8 +137,9 @@ import SwiftUI |
| watches[id] = watch |
| lastLevel[id] = sample.level |
| let model = peer(id) |
| - let fresh = notices.filter { !model.notices.contains($0) } |
| - model.show(state: state, speaking: watch.speaking, notices: notices) |
| + var fresh = notices.filter { !model.notices.contains($0) } |
| + if let connecting, model.connecting != connecting { fresh.append(connecting) } |
| + model.show(state: state, speaking: watch.speaking, notices: notices, connecting: connecting) |
| model.glow.update(level: sample.level, speaking: watch.speaking, now: now) |
| if state == .noSound, before != .noSound { AudioAnnouncer.say(AudioWords.noSound(from: name)) } |
| for line in fresh { AudioAnnouncer.say(line) } |
| @@ -224,6 +230,9 @@ public struct TileAudioInfo: Equatable { |
| /// Lines above the name: their notices about you, or your microphone |
| /// warning. |
| public var notices: [String] |
| + /// "<name> is connecting and can't hear you yet" (spec §15.24), with its |
| + /// moving ellipsis; nil when not said. |
| + public var connecting: String? = nil |
| /// The ⓘ's text, when a line has one. |
| public var detail: String? |
| @@ -232,6 +241,7 @@ public struct TileAudioInfo: Equatable { |
| var out: [String] = [] |
| if case .noSound(let name) = status { out.append(AudioWords.noSound(from: name)) } |
| out.append(contentsOf: notices) |
| + if let connecting { out.append(connecting) } |
| return out |
| } |
| @@ -251,7 +261,7 @@ public struct TileAudioInfo: Equatable { |
| case .meter: status = .live |
| } |
| return TileAudioInfo( |
| - status: status, speaking: audio.speaking, notices: audio.notices, |
| + status: status, speaking: audio.speaking, notices: audio.notices, connecting: audio.connecting, |
| detail: audio.state == .noSound ? AudioWords.noSoundDetail : nil) |
| } |
| } |
| @@ -338,15 +348,55 @@ public struct TileAudioNotices: View { |
| let info: TileAudioInfo? |
| public init(_ info: TileAudioInfo?) { self.info = info } |
| public var body: some View { |
| - if let info, !info.notices.isEmpty { |
| + if let info, !info.notices.isEmpty || info.connecting != nil { |
| VStack(alignment: .leading, spacing: 4) { |
| ForEach(info.notices, id: \.self) { line in |
| AudioLine( |
| text: line, systemImage: "exclamationmark.triangle.fill", |
| detail: line == AudioWords.micSilent ? info.detail : nil) |
| } |
| + if let connecting = info.connecting { |
| + AudioLine(text: connecting, systemImage: "ear", ellipsis: true) |
| + .accessibilityIdentifier("connectingNotice") |
| + } |
| + } |
| + } |
| + } |
| +} |
| + |
| +/// "<name> is connecting and can't hear you yet" and three dots after it |
| +/// (spec §15.24) that brighten in turn — waiting, not wrong — or one steady |
| +/// ellipsis under Reduce Motion. One run of text, so a long name wraps the |
| +/// dots with the words. VoiceOver hears the sentence, never the dots. |
| +public struct ConnectingText: View { |
| + let words: String |
| + @Environment(\.accessibilityReduceMotion) private var reduceMotion |
| + public init(_ words: String) { self.words = words } |
| + public var body: some View { |
| + Group { |
| + if reduceMotion { |
| + Text(words + "…") |
| + } else { |
| + TimelineView(.periodic(from: .now, by: 0.1)) { context in |
| + let t = context.date.timeIntervalSinceReferenceDate |
| + (0..<3).reduce(Text(words)) { line, dot in |
| + line + Text(".").foregroundColor(.white.opacity(Self.opacity(at: t, dot: dot))) |
| + } |
| + } |
| } |
| } |
| + .accessibilityLabel(words) |
| + } |
| + |
| + /// Each dot's brightness at `t`: dim, lit at its turn in a 1.4 s cycle, |
| + /// a fifth of a second after the one before — the browser's |
| + /// `calls-ellipsis` keyframes. |
| + static func opacity(at t: Double, dot: Int) -> Double { |
| + let cycle = 1.4 |
| + let phase = (t - Double(dot) * 0.2).truncatingRemainder(dividingBy: cycle) |
| + let p = (phase < 0 ? phase + cycle : phase) / cycle |
| + let lift = p <= 0.4 ? p / 0.4 : p <= 0.8 ? (0.8 - p) / 0.4 : 0 |
| + return 0.35 + 0.65 * lift |
| } |
| } |
| @@ -357,12 +407,17 @@ struct AudioLine: View { |
| let text: String |
| let systemImage: String |
| var detail: String? |
| + /// The connecting notice's moving dots after the words (spec §15.24). |
| + var ellipsis = false |
| @State private var showing = false |
| var body: some View { |
| HStack(spacing: 5) { |
| Image(systemName: systemImage) |
| - Text(text).lineLimit(2).multilineTextAlignment(.leading) |
| - .fixedSize(horizontal: false, vertical: true) |
| + Group { |
| + if ellipsis { ConnectingText(text) } else { Text(text) } |
| + } |
| + .lineLimit(2).multilineTextAlignment(.leading) |
| + .fixedSize(horizontal: false, vertical: true) |
| if let detail { |
| Button { |
| showing.toggle() |
diff --git a/native/apple/Sources/CallsKit/AudioWork.swift b/native/apple/Sources/CallsKit/AudioWork.swift| index ccf38ec..15e0a36 100644 |
| --- a/native/apple/Sources/CallsKit/AudioWork.swift |
| +++ b/native/apple/Sources/CallsKit/AudioWork.swift |
| @@ -131,6 +131,12 @@ enum PeerCalls { |
| } |
| } |
| + static func statistics(_ pc: RTCPeerConnection, for sender: RTCRtpSender) async -> RTCStatisticsReport { |
| + await withCheckedContinuation { c in |
| + AudioWork.queue.async { pc.statistics(for: sender) { c.resume(returning: $0) } } |
| + } |
| + } |
| + |
| /// Anything else, read or done on the audio queue. |
| static func read<T>(_ pc: RTCPeerConnection, _ body: @escaping (RTCPeerConnection) -> T) async -> T { |
| await AudioWork.run { body(pc) } |
diff --git a/native/apple/Sources/CallsKit/Call.swift b/native/apple/Sources/CallsKit/Call.swift| index 56f4107..801044d 100644 |
| --- a/native/apple/Sources/CallsKit/Call.swift |
| +++ b/native/apple/Sources/CallsKit/Call.swift |
| @@ -87,6 +87,17 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| private var reconnectWatch = ReconnectWatch() |
| @Published public private(set) var reconnecting = false |
| @Published public private(set) var reconnectingPeers: Set<String> = [] |
| + /// "<name> is connecting and can't hear you yet" (spec §15.24): the |
| + /// watch, and the people it is said for now — on their tiles, and as a |
| + /// dot in the mini window when they are not the one it shows. |
| + private var connectingWatch = ConnectingWatch() |
| + @Published public private(set) var connectingPeers: Set<String> = [] |
| + /// This app's own half: whose audio path is established (HearingRule), |
| + /// per connection generation; whose receiver reports have shown this |
| + /// app's audio reaching them; and the list last reported as `hearing`. |
| + private var heardFrom: [String: (epoch: Int64, heard: Bool)] = [:] |
| + private var ackedBy: [String: Int64] = [:] |
| + private var reportedHearing: [String] = [] |
| /// The phase the notices are drawn for: the core's, except that its |
| /// `reconnecting` is only said once ReconnectWatch says so, and is said |
| /// for a second after it is over. |
| @@ -519,6 +530,7 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| private func watchLevels() { |
| levelWatch?.cancel() |
| levels.reset() |
| + resetConnecting() |
| reportedNoSound = [] |
| audioBytes = [:] |
| audioArriving = [:] |
| @@ -544,11 +556,21 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| screenReceived = [:] |
| screenReceivedMoving = [:] |
| levels.reset() |
| + resetConnecting() |
| reportedNoSound = [] |
| audioBytes = [:] |
| audioArriving = [:] |
| } |
| + private func resetConnecting() { |
| + connectingWatch = ConnectingWatch() |
| + if !connectingPeers.isEmpty { connectingPeers = [] } |
| + heardFrom = [:] |
| + ackedBy = [:] |
| + // The core starts every report with `hearing: []` (spec §15.24). |
| + reportedHearing = [] |
| + } |
| + |
| private func sampleMic(now: Double) { |
| let denoiser = media.audio.denoiser |
| let sent = denoiser.level.take() |
| @@ -575,30 +597,79 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| } |
| private func samplePeers(stats: Bool, now: Double) async { |
| - let readings = await media.peerAudio(stats: stats) |
| + // §15.24: the people whose audio path is not established yet are read |
| + // closely, five times a second, until it is; the people in their first |
| + // moments who do not say who they hear are asked about this app's audio. |
| + let mine = state?.peer ?? "" |
| + let joining = connectingWatch.joining |
| + var detail = Set<String>() |
| + var acks = Set<String>() |
| + for person in state?.peers ?? [] where person.peer != mine { |
| + let epoch = person.gen ?? 0 |
| + if heardFrom[person.peer].map({ $0.epoch != epoch || !$0.heard }) ?? true { detail.insert(person.peer) } |
| + if joining.contains(person.peer), HearingRule.hearsYou(person.media, me: mine) == nil, |
| + ackedBy[person.peer] != epoch |
| + { |
| + acks.insert(person.peer) |
| + } |
| + } |
| + let readings = await media.peerAudio(stats: stats, detail: detail, acks: acks) |
| guard levelWatch != nil, let state else { return } |
| let me = state.peer |
| var present = Set<String>() |
| var people: [String: ReconnectWatch.Peer] = [:] |
| + var arrivals: [String: ConnectingWatch.Peer] = [:] |
| for person in state.peers where person.peer != me { |
| guard let reading = readings[person.peer] else { continue } |
| present.insert(person.peer) |
| - if let bytes = reading.bytes { |
| + if stats, let bytes = reading.bytes { |
| audioArriving[person.peer] = bytes > (audioBytes[person.peer] ?? 0) |
| audioBytes[person.peer] = bytes |
| } |
| - people[person.peer] = ReconnectWatch.Peer( |
| - connectionBad: reading.connectionBad, |
| - audioMissing: audioArriving[person.peer] == false && person.media?.muted != true) |
| - var notices = AudioWords.notices(name: person.name, media: person.media, me: me, muted: muted) |
| - if reconnectingPeers.contains(person.peer) { notices.insert(ReconnectWords.peer(person.name), at: 0) } |
| + let epoch = person.gen ?? 0 |
| + var was = heardFrom[person.peer] ?? (epoch, false) |
| + if was.epoch != epoch { was = (epoch, false) } |
| + let heard = HearingRule.heard( |
| + was: was.heard, |
| + .init( |
| + connected: reading.connected, bad: reading.gone, hasReceiver: reading.hasReceiver, |
| + receiverLive: reading.receiverLive, packets: reading.packets ?? 0, decoded: reading.buffers, |
| + senderMuted: person.media?.muted == true)) |
| + heardFrom[person.peer] = (epoch, heard) |
| + if reading.acked == true { ackedBy[person.peer] = epoch } |
| + arrivals[person.peer] = ConnectingWatch.Peer( |
| + epoch: epoch, hearsMe: HearingRule.hearsYou(person.media, me: me), reported: person.media != nil, |
| + connected: reading.connected, acked: ackedBy[person.peer] == epoch) |
| + } |
| + for id in Array(heardFrom.keys) where !present.contains(id) { heardFrom.removeValue(forKey: id) } |
| + for id in Array(ackedBy.keys) where !present.contains(id) { ackedBy.removeValue(forKey: id) } |
| + for id in connectingWatch.observe(at: now, people: arrivals) { |
| + CallLog.info("\(id.prefix(6)) can hear us") |
| + AudioAnnouncer.say(ConnectingWords.canHear(name(of: id))) |
| + } |
| + let nowJoining = connectingWatch.joining |
| + if connectingPeers != connectingWatch.shown { connectingPeers = connectingWatch.shown } |
| + for person in state.peers where person.peer != me { |
| + guard let reading = readings[person.peer] else { continue } |
| + // Someone in their first moments has not lost anything yet: §15.24 |
| + // speaks for them until it gives up. |
| + people[person.peer] = |
| + nowJoining.contains(person.peer) |
| + ? ReconnectWatch.Peer() |
| + : ReconnectWatch.Peer( |
| + connectionBad: reading.connectionBad, |
| + audioMissing: audioArriving[person.peer] == false && person.media?.muted != true) |
| + let said = ConnectingWatch.tile( |
| + name: person.name, media: person.media, me: me, muted: muted, |
| + joining: nowJoining.contains(person.peer), shown: connectingPeers.contains(person.peer), |
| + reconnecting: reconnectingPeers.contains(person.peer)) |
| remotePeakSinceSample = max(remotePeakSinceSample, reading.peak) |
| levels.updatePeer( |
| person.peer, name: person.name, |
| sample: PeerAudioSample( |
| reported: person.media, connected: reading.connected, |
| arriving: audioArriving[person.peer] ?? false, level: reading.peak), |
| - notices: notices, |
| + notices: said.notices, connecting: said.connecting, |
| now: now) |
| } |
| watchReconnect(now: now, people: people) |
| @@ -608,6 +679,16 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| audioArriving.removeValue(forKey: id) |
| } |
| reportNoSound(levels.noSoundFrom) |
| + reportHearing(heardFrom.filter { $0.value.heard }.keys.sorted()) |
| + } |
| + |
| + /// §15.24: whose audio this app is receiving and playing. Sent only on a |
| + /// change, in order, off the main thread, as `noSoundFrom` is. |
| + private func reportHearing(_ peers: [String]) { |
| + guard peers != reportedHearing, let client = bridge else { return } |
| + reportedHearing = peers |
| + let body = json(peers) ?? "[]" |
| + noSoundQueue.async { try? client.setHearing(body) } |
| } |
| /// One look by ReconnectWatch (spec §15.23), and the words it says. |
| @@ -1052,6 +1133,11 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| values["remoteScreens"] = UInt64(remoteScreens.count) |
| values["screens"] = UInt64(screens.count) |
| values["invalidEvents"] = invalidEvents |
| + // §15.24: people still in their first moments, people whose tiles say |
| + // they cannot hear you yet, and how many this app reports it hears. |
| + values["connectingJoining"] = UInt64(connectingWatch.joining.count) |
| + values["connectingShown"] = UInt64(connectingPeers.count) |
| + values["hearing"] = UInt64(reportedHearing.count) |
| // This app's own screen, as it leaves: the slowest connection's. |
| if let screen = screenReport { |
| values["screenSendWidth"] = UInt64(screen.width) |
| @@ -1156,6 +1242,24 @@ extension Call { |
| self.sharingScreen = sharingScreen |
| } |
| + /// Spec §15.24 for the snapshots: these people have just joined and the |
| + /// grace has passed, so each tile says what it would — "connecting", or |
| + /// a report of theirs that replaces it. |
| + @_spi(Fixtures) public func showFixtureConnecting(_ peers: [String]) { |
| + let me = state?.peer ?? "" |
| + var shown: Set<String> = [] |
| + for person in state?.peers ?? [] where peers.contains(person.peer) { |
| + let said = ConnectingWatch.tile( |
| + name: person.name, media: person.media, me: me, muted: muted, joining: true, shown: true, |
| + reconnecting: false) |
| + if said.connecting != nil { shown.insert(person.peer) } |
| + levels.peer(person.peer).show( |
| + state: person.media?.muted == true ? .muted : .meter(0), speaking: false, notices: said.notices, |
| + connecting: said.connecting) |
| + } |
| + connectingPeers = shown |
| + } |
| + |
| /// Everyone's glow, and yours, held at one level, for the snapshots. |
| @_spi(Fixtures) public func showFixtureSpeaking(level: Float) { |
| levels.mic.glow.showFixture(level: level) |
diff --git a/native/apple/Sources/CallsKit/CallFixtures.swift b/native/apple/Sources/CallsKit/CallFixtures.swift| index 071b94b..f09378d 100644 |
| --- a/native/apple/Sources/CallsKit/CallFixtures.swift |
| +++ b/native/apple/Sources/CallsKit/CallFixtures.swift |
| @@ -16,6 +16,9 @@ import SwiftUI |
| case screen |
| /// Somebody waiting to be let in, and the connection coming back. |
| case notices |
| + /// Two people who have just joined (spec §15.24): one still connecting |
| + /// and unable to hear you yet, the other's browser blocking sound. |
| + case connecting |
| public static let title = "Call, 24 Sep 10:21" |
| @@ -41,6 +44,11 @@ import SwiftUI |
| call.showFixture( |
| title: Self.title, status: "Reconnecting…", |
| state: roster("reconnecting", [me, ana], waiting: [knock]), muted: true, floor: true) |
| + case .connecting: |
| + let sam = #"{"peer":"sam","name":"Samuel Okafor","kind":"member","media":{"muted":false,"cameraOff":false,"hearing":[]}}"# |
| + let bea = #"{"peer":"bea","name":"Bea Ruiz","kind":"guest","media":{"muted":false,"cameraOff":true,"audioBlocked":true,"hearing":[]}}"# |
| + call.showFixture(title: Self.title, state: roster("admitted", [me, ana, sam, bea])) |
| + call.showFixtureConnecting(["sam", "bea"]) |
| } |
| } |
| } |
diff --git a/native/apple/Sources/CallsKit/ConnectingWatch.swift b/native/apple/Sources/CallsKit/ConnectingWatch.swift| new file mode 100644 |
| index 0000000..f8827b4 |
| --- /dev/null |
| +++ b/native/apple/Sources/CallsKit/ConnectingWatch.swift |
| @@ -0,0 +1,160 @@ |
| +import Foundation |
| + |
| +/// "<name> is connecting and can't hear you yet" (spec §15.24, 2026-10-01). |
| +/// |
| +/// Paul asked for Zoom's "… is connecting to audio and cannot hear you yet": |
| +/// the gap after someone joins, before their side is receiving and playing |
| +/// your audio, said rather than talked into. Their own report's `hearing` |
| +/// is the authority; a client that does not send one is judged on what this |
| +/// app can see of its own audio reaching them. |
| +/// |
| +/// For each person from the moment they appear in the roster (or appear |
| +/// again, or are given a new connection): |
| +/// |
| +/// - the words are said only once `grace` (1 s) has passed without their |
| +/// hearing you, so an ordinary join says nothing; |
| +/// - they go the moment their report lists you in `hearing` — "Samuel can |
| +/// hear you now"; |
| +/// - for a client that does not say who it hears, once the connection is up |
| +/// and either their receiver reports show your audio reaching them or |
| +/// they have made a report without the list; |
| +/// - and never past `cap` (20 s), after which §15.20's and §15.23's words |
| +/// speak for any problem. |
| +/// |
| +/// Your own mute does not matter: it is about their hearing you when you |
| +/// speak. Pure: fed the facts as they are, with the time. The browser's |
| +/// `connectingStep` is its twin. |
| +struct ConnectingWatch: Equatable { |
| + static let grace = 1.0 |
| + static let cap = 20.0 |
| + |
| + /// One person as they are now. |
| + struct Peer: Equatable { |
| + /// The pair generation: a new one is a new connection, and they cannot |
| + /// hear you on it yet. |
| + var epoch: Int64 = 0 |
| + /// Their report on you (`HearingRule.hearsYou`): listed, not listed, |
| + /// or nil for no list at all. |
| + var hearsMe: Bool? |
| + /// They have made a media report. |
| + var reported = false |
| + /// The connection to them is up. |
| + var connected = false |
| + /// Their receiver reports say your audio reaches them. |
| + var acked = false |
| + } |
| + |
| + private struct Record: Equatable { |
| + var epoch: Int64 |
| + var since: Double |
| + var done = false |
| + var shown = false |
| + } |
| + private var records: [String: Record] = [:] |
| + |
| + init() {} |
| + |
| + /// People still in their first moments, whether or not the words show yet. |
| + var joining: Set<String> { Set(records.filter { !$0.value.done }.keys) } |
| + /// People whose tiles say it. |
| + var shown: Set<String> { Set(records.filter { $0.value.shown }.keys) } |
| + |
| + /// One look at everyone but you. Returns the people whose words have just |
| + /// gone because they can hear you, for "… can hear you now". |
| + @discardableResult |
| + mutating func observe(at now: Double, people: [String: Peer]) -> [String] { |
| + for id in Array(records.keys) where people[id] == nil { records.removeValue(forKey: id) } |
| + var cleared: [String] = [] |
| + for (id, p) in people { |
| + var r = records[id] ?? Record(epoch: p.epoch, since: now) |
| + if r.epoch != p.epoch { r = Record(epoch: p.epoch, since: now) } |
| + if !r.done { |
| + let fallback = p.hearsMe == nil && p.connected && (p.acked || p.reported) |
| + if p.hearsMe == true || fallback { |
| + if r.shown { cleared.append(id) } |
| + r.done = true |
| + r.shown = false |
| + } else if now - r.since >= Self.cap { |
| + r.done = true |
| + r.shown = false |
| + } else { |
| + r.shown = now - r.since >= Self.grace |
| + } |
| + } |
| + records[id] = r |
| + } |
| + return cleared.sorted() |
| + } |
| + |
| + /// What a person's tile says about them and you, in order (the browser's |
| + /// `peerNotice`): in their first moments, their own more specific report |
| + /// (their browser blocking sound, or no sound from you) and otherwise the |
| + /// connecting words once the grace has passed — "Reconnecting" would be |
| + /// wrong of a connection that has never been up; after that, §15.23's |
| + /// "Reconnecting to …" first and then their reports. `connecting` is the |
| + /// line that carries the moving ellipsis, and is never set beside |
| + /// `notices`. |
| + static func tile( |
| + name: String, media: PeerMedia?, me: String, muted: Bool, joining: Bool, shown: Bool, |
| + reconnecting: Bool |
| + ) -> (notices: [String], connecting: String?) { |
| + let reports = AudioWords.notices(name: name, media: media, me: me, muted: muted) |
| + if joining { |
| + if !reports.isEmpty { return (reports, nil) } |
| + return ([], shown ? ConnectingWords.connecting(name) : nil) |
| + } |
| + return ((reconnecting ? [ReconnectWords.peer(name)] : []) + reports, nil) |
| + } |
| +} |
| + |
| +/// This app's half of §15.24: whose audio it can say it hears. |
| +enum HearingRule { |
| + /// One connection's audio as it is now. |
| + struct Facts: Equatable { |
| + /// ICE connected or completed. |
| + var connected = false |
| + /// ICE failed or closed: the path is gone. |
| + var bad = false |
| + var hasReceiver = false |
| + /// The audio receiver's track is live. |
| + var receiverLive = false |
| + /// Audio packets received on it so far. |
| + var packets: UInt64 = 0 |
| + /// Decoded audio frames that reached the receive tap since the last look. |
| + var decoded = 0 |
| + /// They say they are muted (or have no microphone). |
| + var senderMuted = false |
| + } |
| + |
| + /// Whether the audio path from someone is established. Once it is, it |
| + /// stays so until the connection fails or closes. Established is: the |
| + /// connection up, and either their audio arriving — packets received and |
| + /// decoded frames coming out — on a live receiver, or, for someone who |
| + /// says they are muted, nothing more than that: a muted sender may send |
| + /// only silence, or no audio at all, and must not look unheard, because |
| + /// the moment they unmute, it plays. The browser's `judgeHeard` is its |
| + /// twin. A native app plays whatever it decodes: there is no autoplay |
| + /// policy to refuse it. |
| + static func heard(was: Bool, _ f: Facts) -> Bool { |
| + if f.bad { return false } |
| + if was { return true } |
| + if !f.connected { return false } |
| + if f.senderMuted, !f.hasReceiver || f.receiverLive { return true } |
| + return f.receiverLive && f.packets > 0 && f.decoded > 0 |
| + } |
| + |
| + /// A person's report on you: true when their `hearing` lists you, false |
| + /// when it has a list and you are not on it, nil when it has no list — no |
| + /// report yet, or a client from before §15.24. |
| + static func hearsYou(_ media: PeerMedia?, me: String) -> Bool? { |
| + guard let list = media?.hearing else { return nil } |
| + return !me.isEmpty && list.contains(me) |
| + } |
| +} |
| + |
| +/// The words (spec §15.24). The ellipsis is drawn apart from them |
| +/// (`ConnectingEllipsis`), so VoiceOver hears the sentence and not the dots. |
| +public enum ConnectingWords { |
| + public static func connecting(_ name: String) -> String { "\(name) is connecting and can’t hear you yet" } |
| + public static func canHear(_ name: String) -> String { "\(name) can hear you now" } |
| +} |
diff --git a/native/apple/Sources/CallsKit/Media.swift b/native/apple/Sources/CallsKit/Media.swift| index 92857a4..b96c140 100644 |
| --- a/native/apple/Sources/CallsKit/Media.swift |
| +++ b/native/apple/Sources/CallsKit/Media.swift |
| @@ -386,32 +386,66 @@ import LiveKitWebRTC |
| /// Each remote person's sound, for the meters (spec §15.20): the level |
| /// their receive tap has seen since the last look, whether the |
| - /// connection is up, and — when `stats` is set — the audio bytes |
| - /// received so far, one `statistics(for:)` per audio receiver rather |
| - /// than the whole report. Nil bytes when not asked. |
| - func peerAudio(stats: Bool) async -> [String: PeerAudioReading] { |
| + /// connection is up, and — when `stats` is set, or for the people in |
| + /// `detail` — the audio bytes and packets received so far and whether |
| + /// the receiver is live, one `statistics(for:)` per audio receiver rather |
| + /// than the whole report. For the people in `acks`, whether their |
| + /// receiver reports say this app's audio reaches them (spec §15.24's |
| + /// fallback for a client that does not say who it hears). |
| + func peerAudio(stats: Bool, detail: Set<String> = [], acks: Set<String> = []) async -> [String: PeerAudioReading] { |
| var out: [String: PeerAudioReading] = [:] |
| for (id, peer) in peers { |
| guard let pc = peer.pc else { continue } |
| let (peak, buffers) = peer.tap.take() |
| + let deep = stats || detail.contains(id) |
| + let ack = acks.contains(id) |
| // Read on the audio queue, as every call into the connection is |
| // (PeerCalls): five times a second, and each one a blocking hop to |
| // WebRTC's signalling thread. |
| - let (state, receivers) = await PeerCalls.read(pc) { pc in |
| - (pc.iceConnectionState, stats ? pc.receivers.filter { $0.track?.kind == "audio" } : []) |
| + let (state, receivers, senders) = await PeerCalls.read(pc) { pc in |
| + ( |
| + pc.iceConnectionState, deep ? pc.receivers.filter { $0.track?.kind == "audio" } : [], |
| + ack ? pc.senders.filter { $0.track?.kind == "audio" } : [] |
| + ) |
| } |
| var reading = PeerAudioReading( |
| peak: peak, buffers: buffers, connected: state == .connected || state == .completed, |
| connectionBad: state == .disconnected || state == .failed) |
| - if stats { |
| + reading.gone = state == .failed || state == .closed |
| + if deep { |
| var bytes: UInt64 = 0 |
| + var packets: UInt64 = 0 |
| + var live = false |
| for receiver in receivers { |
| + if receiver.track?.readyState == .live { live = true } |
| let report = await PeerCalls.statistics(pc, for: receiver) |
| for stat in report.statistics.values where stat.type == "inbound-rtp" { |
| bytes += (stat.values["bytesReceived"] as? NSNumber)?.uint64Value ?? 0 |
| + packets += (stat.values["packetsReceived"] as? NSNumber)?.uint64Value ?? 0 |
| } |
| } |
| reading.bytes = bytes |
| + reading.packets = packets |
| + reading.hasReceiver = !receivers.isEmpty |
| + reading.receiverLive = live |
| + } |
| + if ack { |
| + var sent: UInt64 = 0 |
| + var acked = false |
| + for sender in senders { |
| + let report = await PeerCalls.statistics(pc, for: sender) |
| + for stat in report.statistics.values { |
| + if stat.type == "outbound-rtp" { |
| + sent += (stat.values["packetsSent"] as? NSNumber)?.uint64Value ?? 0 |
| + } else if stat.type == "remote-inbound-rtp" { |
| + // Exists only once a receiver report has come back. |
| + let received = (stat.values["packetsReceived"] as? NSNumber)?.uint64Value ?? 0 |
| + let measured = (stat.values["roundTripTimeMeasurements"] as? NSNumber)?.uint64Value ?? 0 |
| + if received > 0 || measured > 0 || stat.values["roundTripTime"] != nil { acked = true } |
| + } |
| + } |
| + } |
| + reading.acked = sent > 0 && acked |
| } |
| out[id] = reading |
| } |
| @@ -464,7 +498,17 @@ struct PeerAudioReading: Equatable, Sendable { |
| var connected: Bool |
| /// ICE `disconnected` or `failed` (spec §15.23's three seconds). |
| var connectionBad = false |
| + /// ICE `failed` or `closed`: whatever was heard on it is gone (§15.24). |
| + var gone = false |
| var bytes: UInt64? |
| + /// Audio packets received, the receiver there at all, and its track |
| + /// live (§15.24); read with `bytes`. |
| + var packets: UInt64? |
| + var hasReceiver = false |
| + var receiverLive = false |
| + /// Their receiver reports say this app's audio reaches them; nil when |
| + /// not asked. |
| + var acked: Bool? |
| } |
| @MainActor private final class MediaPeer: NSObject, RTCPeerConnectionDelegate { |
diff --git a/native/apple/Sources/CallsKit/MiniWindow.swift b/native/apple/Sources/CallsKit/MiniWindow.swift| index 9b819b0..d4af785 100644 |
| --- a/native/apple/Sources/CallsKit/MiniWindow.swift |
| +++ b/native/apple/Sources/CallsKit/MiniWindow.swift |
| @@ -422,6 +422,13 @@ |
| return item |
| } |
| + /// The people still connecting who are not the one on show. |
| + private var offStage: [String] { |
| + var shown: String? |
| + if case .camera(let peer) = subject { shown = peer } |
| + return call.connectingPeers.filter { $0 != shown }.sorted() |
| + } |
| + |
| var body: some View { |
| ZStack { |
| Group { |
| @@ -455,6 +462,10 @@ |
| .accessibilityLabel("Your microphone is muted") |
| } |
| Spacer() |
| + // Spec §15.24, for the people this window is not showing: the |
| + // one it shows says it on its own tile. Top right, clear of |
| + // the tile's Muted badge. |
| + ConnectingDot(names: offStage.map { call.name(of: $0) }) |
| if showingControls { |
| Button("Collapse to a tab", glyph: "arrow.down.right.and.arrow.up.left") { |
| actions.collapse() |
| @@ -513,6 +524,17 @@ |
| .background( |
| call.muted ? Color(red: 0.65, green: 0.30, blue: 0.02) : CallsAppearance.accent, |
| in: Circle()) |
| + // Spec §15.24: someone is still connecting and cannot hear you |
| + // yet. A dot only — the tab has no room for words; its |
| + // accessibility label and the window, opened, say who. |
| + .overlay(alignment: .topTrailing) { |
| + if !call.connectingPeers.isEmpty { |
| + Circle().fill(ConnectingDot.colour).frame(width: 8, height: 8) |
| + .overlay(Circle().strokeBorder(Color.black.opacity(0.35), lineWidth: 1)) |
| + .offset(x: 2, y: -2) |
| + .help(call.connectingPeers.sorted().map { ConnectingWords.connecting(call.name(of: $0)) }.joined(separator: "\n")) |
| + } |
| + } |
| Text(caption).font(.callout.weight(.medium)).lineLimit(1).truncationMode(.tail) |
| .foregroundStyle(.white) |
| Spacer(minLength: 0) |
| @@ -531,7 +553,9 @@ |
| .clipShape(Capsule()) |
| .accessibilityElement(children: .ignore) |
| .accessibilityAddTraits(.isButton) |
| - .accessibilityLabel("Calls, \(caption)") |
| + .accessibilityLabel( |
| + (["Calls, \(caption)"] + call.connectingPeers.sorted().map { ConnectingWords.connecting(call.name(of: $0)) }) |
| + .joined(separator: ". ")) |
| .accessibilityValue(call.muted ? "Microphone muted" : "Microphone on") |
| .accessibilityHint("Shows the mini window.") |
| .accessibilityAction { actions.expand() } |
| @@ -543,6 +567,33 @@ |
| } |
| } |
| + /// The mini window's whole say on spec §15.24: a small ear on the glass |
| + /// while someone it is not showing is connecting and cannot hear you yet, |
| + /// with their names in its tooltip and for VoiceOver. Nothing at all |
| + /// otherwise. |
| + struct ConnectingDot: View { |
| + /// A warm amber, light enough to tell from the muted microphone's |
| + /// brown on the tab. |
| + static let colour = Color(red: 1, green: 0.78, blue: 0.42) |
| + let names: [String] |
| + var body: some View { |
| + if !names.isEmpty { |
| + let words = names.map(ConnectingWords.connecting).joined(separator: "\n") |
| + Image(systemName: "ear") |
| + .font(.callout.bold()).foregroundStyle(.white) |
| + .frame(width: 32, height: 32) |
| + .callOverlaySurface(Circle(), rim: false) |
| + .overlay(alignment: .topTrailing) { |
| + Circle().fill(Self.colour).frame(width: 7, height: 7) |
| + } |
| + .help(words) |
| + .accessibilityElement(children: .ignore) |
| + .accessibilityLabel(words) |
| + .accessibilityIdentifier("connectingDot") |
| + } |
| + } |
| + } |
| + |
| /// The window's background: a click, a drag that moves the window, and |
| /// — for the expanded window — a drag from an edge or corner that |
| /// resizes it. Handled here rather than with isMovableByWindowBackground |
diff --git a/native/apple/Sources/CallsKit/Wire.swift b/native/apple/Sources/CallsKit/Wire.swift| index 2c20d31..d750c01 100644 |
| --- a/native/apple/Sources/CallsKit/Wire.swift |
| +++ b/native/apple/Sources/CallsKit/Wire.swift |
| @@ -24,16 +24,21 @@ public struct PeerMedia: Decodable, Equatable, Sendable { |
| /// The peers whose shared screen has stopped moving for it (§15.21). |
| /// A sharer that finds itself here recovers its screen. Nil is none. |
| public let screenStalled: [String]? |
| + /// The peers whose audio it is actually receiving and playing (§15.24). |
| + /// Nil (absent) is a client that does not say; empty is one that hears |
| + /// nobody yet. |
| + public let hearing: [String]? |
| public init( |
| muted: Bool, cameraOff: Bool, audioBlocked: Bool? = nil, noSoundFrom: [String]? = nil, |
| - screenStalled: [String]? = nil |
| + screenStalled: [String]? = nil, hearing: [String]? = nil |
| ) { |
| self.muted = muted |
| self.cameraOff = cameraOff |
| self.audioBlocked = audioBlocked |
| self.noSoundFrom = noSoundFrom |
| self.screenStalled = screenStalled |
| + self.hearing = hearing |
| } |
| } |
| public struct RoomState: Decodable { |
diff --git a/native/apple/Tests/CallsKitTests/ConnectingWatchTests.swift b/native/apple/Tests/CallsKitTests/ConnectingWatchTests.swift| new file mode 100644 |
| index 0000000..53e29ae |
| --- /dev/null |
| +++ b/native/apple/Tests/CallsKitTests/ConnectingWatchTests.swift |
| @@ -0,0 +1,231 @@ |
| +import XCTest |
| + |
| +@testable import CallsKit |
| + |
| +/// "<name> is connecting and can't hear you yet" (spec §15.24). Paul, |
| +/// 2026-10-01, asking for Zoom's "… is connecting to audio and cannot hear |
| +/// you yet": the gap after someone joins, said rather than talked into. |
| +final class ConnectingWatchTests: XCTestCase { |
| + private typealias Peer = ConnectingWatch.Peer |
| + private let waiting = Peer(epoch: 1, hearsMe: false, reported: true, connected: false, acked: false) |
| + |
| + /// Runs the watch at 5 Hz from `from` to `to` seconds; when Sam's words |
| + /// showed, and when they cleared with "can hear you now". `sam` nil is |
| + /// Sam not in the roster. |
| + private func run( |
| + _ watch: inout ConnectingWatch, from: Double, to: Double, sam: (Double) -> Peer? |
| + ) -> (shown: [Double], cleared: [Double]) { |
| + var shown: [Double] = [] |
| + var cleared: [Double] = [] |
| + var t = from |
| + while t < to { |
| + let people = sam(t).map { ["sam": $0] } ?? [:] |
| + if watch.observe(at: t, people: people).contains("sam") { cleared.append(t) } |
| + if watch.shown.contains("sam") { shown.append(t) } |
| + t = (t * 10 + 2).rounded() / 10 |
| + } |
| + return (shown, cleared) |
| + } |
| + |
| + func testTheTimingsAreTheOnesTheSpecNames() { |
| + XCTAssertEqual(ConnectingWatch.grace, 1) |
| + XCTAssertEqual(ConnectingWatch.cap, 20) |
| + } |
| + |
| + func testAnOrdinaryJoinHeardWithinTheGraceSaysNothing() { |
| + var watch = ConnectingWatch() |
| + var sam = waiting |
| + let seen = run(&watch, from: 0, to: 10) { t in |
| + sam.connected = t >= 0.4 |
| + sam.hearsMe = t >= 0.8 |
| + return sam |
| + } |
| + XCTAssertEqual(seen.shown, []) |
| + XCTAssertEqual(seen.cleared, [], "nothing was said, so \"can hear you now\" is not said either") |
| + XCTAssertFalse(watch.joining.contains("sam")) |
| + } |
| + |
| + func testASlowJoinIsSaidAfterTheGraceAndClearsWhenTheyHearYou() { |
| + var watch = ConnectingWatch() |
| + var sam = waiting |
| + let seen = run(&watch, from: 0, to: 12) { t in |
| + sam.connected = t >= 5 |
| + sam.hearsMe = t >= 6 |
| + return sam |
| + } |
| + XCTAssertEqual(seen.shown.first, 1) |
| + XCTAssertEqual(seen.shown.last, 5.8) |
| + XCTAssertEqual(seen.cleared, [6]) |
| + } |
| + |
| + func testOnceClearedItStaysClearedWhateverTheirListDoes() { |
| + var watch = ConnectingWatch() |
| + var sam = waiting |
| + sam.connected = true |
| + let seen = run(&watch, from: 0, to: 30) { t in |
| + sam.hearsMe = t >= 3 && t < 10 |
| + return sam |
| + } |
| + XCTAssertEqual(seen.cleared, [3]) |
| + XCTAssertEqual(seen.shown.last, 2.8) |
| + } |
| + |
| + func testItIsGivenUpAtTheCapSilently() { |
| + var watch = ConnectingWatch() |
| + let seen = run(&watch, from: 0, to: 40) { _ in self.waiting } |
| + XCTAssertEqual(seen.shown.first, 1) |
| + XCTAssertEqual(seen.shown.last, 19.8) |
| + XCTAssertEqual(seen.cleared, [], "a cap is not their hearing you") |
| + XCTAssertTrue(watch.joining.isEmpty) |
| + } |
| + |
| + func testAClientThatDoesNotSayIsLetGoOnItsFirstReportAndAConnection() { |
| + var watch = ConnectingWatch() |
| + let seen = run(&watch, from: 0, to: 10) { t in |
| + Peer(epoch: 1, hearsMe: nil, reported: t >= 4, connected: t >= 2, acked: false) |
| + } |
| + XCTAssertEqual(seen.cleared, [4]) |
| + } |
| + |
| + func testOrOnAConnectionAndReceiverReportsShowingYourAudioReachesThem() { |
| + var watch = ConnectingWatch() |
| + let seen = run(&watch, from: 0, to: 10) { t in |
| + Peer(epoch: 1, hearsMe: nil, reported: false, connected: t >= 2, acked: t >= 3) |
| + } |
| + XCTAssertEqual(seen.cleared, [3]) |
| + var unconnected = ConnectingWatch() |
| + let never = run(&unconnected, from: 0, to: 10) { _ in |
| + Peer(epoch: 1, hearsMe: nil, reported: true, connected: false, acked: true) |
| + } |
| + XCTAssertEqual(never.cleared, [], "never without a connection") |
| + } |
| + |
| + func testAClientThatSaysWhoItHearsIsBelievedOverYourOwnEvidence() { |
| + var watch = ConnectingWatch() |
| + let seen = run(&watch, from: 0, to: 10) { _ in |
| + Peer(epoch: 1, hearsMe: false, reported: true, connected: true, acked: true) |
| + } |
| + XCTAssertEqual(seen.cleared, []) |
| + XCTAssertFalse(seen.shown.isEmpty) |
| + } |
| + |
| + func testARejoinStartsItAgainAndSoDoesANewConnection() { |
| + var watch = ConnectingWatch() |
| + var sam = waiting |
| + let seen = run(&watch, from: 0, to: 20) { t in |
| + if t >= 5 && t < 6 { return nil } |
| + sam.connected = true |
| + sam.hearsMe = t < 5 ? t >= 2 : t >= 9 |
| + return sam |
| + } |
| + XCTAssertEqual(seen.cleared, [2, 9]) |
| + XCTAssertTrue(seen.shown.contains(7)) |
| + XCTAssertFalse(seen.shown.contains(6.8), "the grace again from the moment they are back") |
| + |
| + var rebuilt = ConnectingWatch() |
| + var again = waiting |
| + again.connected = true |
| + let second = run(&rebuilt, from: 0, to: 20) { t in |
| + again.epoch = t < 8 ? 1 : 2 |
| + again.hearsMe = t < 1 || t >= 12 |
| + return again |
| + } |
| + XCTAssertEqual(second.cleared, [12]) |
| + XCTAssertEqual(second.shown.first, 9) |
| + } |
| + |
| + func testAMoreSpecificReportReplacesConnectingAndReconnectingWaitsUntilAfter() { |
| + let plain = PeerMedia(muted: false, cameraOff: false, hearing: []) |
| + func tile( |
| + _ media: PeerMedia, muted: Bool = false, joining: Bool = true, shown: Bool = true, |
| + reconnecting: Bool = false |
| + ) -> (notices: [String], connecting: String?) { |
| + ConnectingWatch.tile( |
| + name: "Sam", media: media, me: "me", muted: muted, joining: joining, shown: shown, |
| + reconnecting: reconnecting) |
| + } |
| + XCTAssertNil(tile(plain, shown: false).connecting, "inside the grace") |
| + XCTAssertEqual(tile(plain).connecting, "Sam is connecting and can’t hear you yet") |
| + XCTAssertEqual(tile(plain).notices, []) |
| + XCTAssertNotNil(tile(plain, muted: true).connecting, "your own mute does not matter") |
| + XCTAssertEqual( |
| + tile(plain, reconnecting: true).notices, [], "a connection that has never been up is not reconnecting") |
| + let blocked = PeerMedia(muted: false, cameraOff: false, audioBlocked: true, hearing: []) |
| + XCTAssertEqual(tile(blocked, shown: false).notices, [AudioWords.blocked("Sam")], "even inside the grace") |
| + XCTAssertNil(tile(blocked).connecting) |
| + let silent = PeerMedia(muted: false, cameraOff: false, noSoundFrom: ["me"], hearing: ["me"]) |
| + XCTAssertEqual(tile(silent).notices, [AudioWords.notHearingYou("Sam")]) |
| + XCTAssertNil(tile(silent).connecting) |
| + XCTAssertNotNil( |
| + tile(PeerMedia(muted: false, cameraOff: false, noSoundFrom: ["me"]), muted: true).connecting, |
| + "no sound from you while you are muted is expected, so it replaces nothing") |
| + XCTAssertEqual( |
| + tile(plain, joining: false, shown: false, reconnecting: true).notices, [ReconnectWords.peer("Sam")]) |
| + XCTAssertEqual( |
| + tile(blocked, joining: false, shown: false, reconnecting: true).notices, |
| + [ReconnectWords.peer("Sam"), AudioWords.blocked("Sam")]) |
| + XCTAssertEqual(tile(plain, joining: false, shown: false).notices, []) |
| + } |
| + |
| + func testHearsYou() { |
| + XCTAssertEqual(HearingRule.hearsYou(PeerMedia(muted: false, cameraOff: false, hearing: ["me"]), me: "me"), true) |
| + XCTAssertEqual(HearingRule.hearsYou(PeerMedia(muted: false, cameraOff: false, hearing: []), me: "me"), false) |
| + XCTAssertNil(HearingRule.hearsYou(PeerMedia(muted: false, cameraOff: false), me: "me")) |
| + XCTAssertNil(HearingRule.hearsYou(nil, me: "me")) |
| + } |
| + |
| + /// This app's own half: when a path counts as heard. A muted sender must |
| + /// not look unheard. |
| + func testHeardIsArrivingAndDecodedOrAMutedSenderOnALiveConnection() { |
| + let base = HearingRule.Facts( |
| + connected: true, bad: false, hasReceiver: true, receiverLive: true, packets: 0, decoded: 0, |
| + senderMuted: false) |
| + func with(_ change: (inout HearingRule.Facts) -> Void) -> HearingRule.Facts { |
| + var f = base |
| + change(&f) |
| + return f |
| + } |
| + XCTAssertFalse(HearingRule.heard(was: false, base), "nothing has arrived") |
| + XCTAssertFalse(HearingRule.heard(was: false, with { $0.packets = 10 }), "packets, nothing decoded") |
| + XCTAssertTrue(HearingRule.heard(was: false, with { $0.packets = 10; $0.decoded = 3 })) |
| + XCTAssertFalse( |
| + HearingRule.heard(was: false, with { $0.packets = 10; $0.decoded = 3; $0.connected = false }), |
| + "not before the connection is up") |
| + XCTAssertTrue(HearingRule.heard(was: false, with { $0.senderMuted = true }), "a muted sender") |
| + XCTAssertTrue( |
| + HearingRule.heard(was: false, with { $0.senderMuted = true; $0.hasReceiver = false; $0.receiverLive = false }), |
| + "nor one with no microphone at all") |
| + XCTAssertFalse( |
| + HearingRule.heard(was: false, with { $0.senderMuted = true; $0.receiverLive = false }), "but not a dead receiver") |
| + XCTAssertTrue(HearingRule.heard(was: true, base), "once heard, kept") |
| + XCTAssertTrue(HearingRule.heard(was: true, with { $0.connected = false }), "a blip is not losing them") |
| + XCTAssertFalse(HearingRule.heard(was: true, with { $0.bad = true }), "a failed connection is") |
| + } |
| + |
| + func testTheHearingListDecodesAndAbsentIsNotEmpty() throws { |
| + func media(_ json: String) throws -> PeerMedia? { |
| + let state = try JSONDecoder().decode( |
| + RoomState.self, |
| + from: Data( |
| + #"{"phase":"admitted","peer":"me","boot":"b","waiting":[],"peers":[{"peer":"sam","name":"Sam","kind":"member","gen":1,"media":\#(json)}]}"# |
| + .utf8)) |
| + return Call.peerMedia("sam", in: state) |
| + } |
| + XCTAssertEqual(try media(#"{"muted":false,"cameraOff":false,"hearing":["me"]}"#)?.hearing, ["me"]) |
| + XCTAssertEqual(try media(#"{"muted":false,"cameraOff":false,"hearing":[]}"#)?.hearing, []) |
| + XCTAssertNil(try media(#"{"muted":false,"cameraOff":false}"#)?.hearing) |
| + } |
| + |
| + func testTheDotsTakeTurnsAndNeverGoOut() { |
| + for dot in 0..<3 { |
| + for step in 0..<28 { |
| + let o = ConnectingText.opacity(at: Double(step) * 0.05, dot: dot) |
| + XCTAssertGreaterThanOrEqual(o, 0.35) |
| + XCTAssertLessThanOrEqual(o, 1.0001) |
| + } |
| + } |
| + XCTAssertEqual(ConnectingText.opacity(at: 0.56, dot: 0), 1, accuracy: 0.001) |
| + XCTAssertEqual(ConnectingText.opacity(at: 0.76, dot: 1), 1, accuracy: 0.001) |
| + XCTAssertEqual(ConnectingText.opacity(at: 0.96, dot: 2), 1, accuracy: 0.001) |
| + } |
| +} |
diff --git a/native/ios/UITests/CallsPhoneUITests.swift b/native/ios/UITests/CallsPhoneUITests.swift| index d9a9a47..92525ab 100644 |
| --- a/native/ios/UITests/CallsPhoneUITests.swift |
| +++ b/native/ios/UITests/CallsPhoneUITests.swift |
| @@ -244,6 +244,30 @@ final class CallsPhoneUITests: XCTestCase { |
| } |
| } |
| + /// Spec §15.24: two people who have just joined — one connecting and |
| + /// unable to hear you yet, the other's browser blocking sound in place of |
| + /// that — in Grid and Speaker, light and dark. The notice is on the tile, |
| + /// and VoiceOver hears it in the tile's label, without the dots. |
| + @MainActor func testConnectingSnapshots() throws { |
| + XCUIDevice.shared.orientation = .portrait |
| + for dark in [false, true] { |
| + let scheme = dark ? "dark" : "light" |
| + for layout in ["grid", "speaker"] { |
| + let app = launchCall( |
| + "connecting", |
| + ["-CallsFixtureControls", "hidden", "-CallsFixtureLayout", layout, "-CallsFixtureOrder", "sam,bea,ana,me"], |
| + dark: dark) |
| + let notice = app.descendants(matching: .any) |
| + .matching(NSPredicate(format: "label CONTAINS %@", "Samuel Okafor is connecting and can’t hear you yet")) |
| + .firstMatch |
| + XCTAssertTrue(notice.waitForExistence(timeout: 15), "Samuel's tile says it") |
| + XCTAssertFalse(notice.label.contains("…"), "and VoiceOver hears no dots") |
| + Thread.sleep(forTimeInterval: 1) |
| + try snapshot(app, "connecting-\(layout)-portrait-\(scheme)") |
| + } |
| + } |
| + } |
| + |
| /// The overlay over black, white and a busy frame, in each of your own |
| /// states, light and dark: it should look the same in both. The |
| /// microphone sits beside the menu here so its muted state shows. |
diff --git a/test/browser/native.mjs b/test/browser/native.mjs| index 9ade7d2..f5a85ca 100644 |
| --- a/test/browser/native.mjs |
| +++ b/test/browser/native.mjs |
| @@ -28,6 +28,20 @@ try { |
| await page.goto(`${origin}/m/${room}`); |
| await page.waitForFunction(() => !!window.meet); |
| console.log('browser before native', await page.evaluate(() => ({status: document.querySelector('#status')?.textContent, tiles: document.querySelectorAll('#tiles video').length}))); |
| + // Spec §15.24, both ways: the browser's offers, answers and candidates |
| + // are held back for the first few seconds, so the app and the browser |
| + // each have someone who cannot hear them yet, and each says so. |
| + const HOLD = 5000; |
| + let releaseAt = 0; |
| + const holdSignals = async (route) => { |
| + const body = route.request().postData() || ''; |
| + if (/"type":"(offer|answer|candidate)"/.test(body)) { |
| + if (!releaseAt) releaseAt = Date.now() + HOLD; |
| + await new Promise(r => setTimeout(r, Math.max(0, releaseAt - Date.now()))); |
| + } |
| + await route.continue(); |
| + }; |
| + if (mode !== 'guest') await page.route('**/signal', holdSignals); |
| native = spawn(binary, [], { stdio: ['pipe', 'pipe', 'pipe'] }); |
| native.stdin.write(process.env.CALLS_NATIVE_INPUT + '\n'); |
| let sample = null; |
| @@ -76,6 +90,33 @@ try { |
| await page.getByRole('button', { name: 'Admit', exact: true }).click(); |
| console.log('PASS guest media waits for member admission'); |
| } |
| + if (mode !== 'guest') { |
| + await wait(() => sample?.peer, 'the app is in the call'); |
| + const app = sample.peer; |
| + const connecting = () => page.evaluate((id) => { |
| + const s = window.meet.state(); |
| + return { c: s.connecting[id] || null, h: s.hearing[id] || null, media: s.remoteMedia[id] || null, |
| + me: s.peer, hearingList: s.hearingList }; |
| + }, app); |
| + await wait(async () => { const c = await connecting(); return c.h && c.h.noticeKind === 'connecting'; }, |
| + 'the browser says the app cannot hear it yet', 15); |
| + const words = (await connecting()).h.notice; |
| + assert.match(words, / is connecting and can’t hear you yet$/); |
| + console.log(`PASS the browser says "${words}" while the connection is held`); |
| + await wait(() => sample?.stats.connectingShown === 1, 'the app says the browser cannot hear it yet', 15); |
| + console.log('PASS the app says the browser cannot hear it yet'); |
| + await wait(async () => { const c = await connecting(); return c.c && !c.c.joining && c.h.notice === ''; }, |
| + 'the browser stops saying it once the app hears it', 30); |
| + const heard = await connecting(); |
| + assert.ok((heard.media.hearing || []).includes(heard.me), 'because the app reports it hears the browser'); |
| + assert.ok(Date.now() >= releaseAt, 'and not before the hold let go'); |
| + console.log('PASS the app reports hearing the browser, and the browser\u2019s notice goes'); |
| + await wait(() => sample?.stats.connectingShown === 0 && sample?.stats.connectingJoining === 0 && sample?.stats.hearing === 1, |
| + 'the app stops saying it once the browser hears it', 30); |
| + assert.ok(heard.hearingList.includes(app) || (await connecting()).hearingList.includes(app), 'the browser reports it hears the app'); |
| + console.log('PASS the browser reports hearing the app, and the app\u2019s notice goes'); |
| + await page.unroute('**/signal', holdSignals); |
| + } |
| await wait(() => sample?.stats.framesDecoded > 15 && sample?.stats.audioBytesReceived > 0 && sample?.stats.videoBytesSent > 0 && sample?.stats.audioBytesSent > 0, 'native receives browser audio/video'); |
| console.log('PASS native decoded browser video and received audio'); |
| // The synthetic microphone runs through WebRTC's capture processing, so |