amadan
Show a shared screen in the native apps
A browser that shared its screen reached the apps as a second video
track, and the apps had one slot per peer for video. So the screen took
the camera's tile — and whichever of the two arrived last was the face
the call showed for that person until one of them renegotiated again.
Receiving is now spec §5.4's, and the browser is the contract for it.
A sharer adds its display as one sendonly transceiver in a stream of its
own, says `share {on:true}`, and stops by stopping the transceiver and
saying `share {on:false}`. Nothing on the wire says which m-line is the
screen, so each receiver works it out, and the app does it with
room.js's classifyTrack ported rule for rule into ScreenShare.swift:
recvonly on a connection that sends a camera, then the camera's mid,
then its stream, then the stream the microphone came in, and the share
message only as the tiebreaker. Its node tests are ported case for case
beside it. A native app and a browser looking at the same sender reach
the same answer, or one of them captions a face as a screen.
The share message draws the tile before the media lands, as it does in
the browser, and either ending takes it away: the message, or the
receiver going with its stopped transceiver. Only the receiver the
screen is showing from counts, for the reason the browser learned in
Codex S4-15. Both endings were checked alone against a real browser,
with the other disabled; the gate now runs them together.
On the stage a screen beats a camera and the newest screen beats an
older one, until somebody chooses otherwise from the participant menu,
which now lists screens too. A screen is letterboxed rather than
cropped, because the edge of a screen is where its menu bar is. On the
Mac the cameras come down to thumbnails beside yours while a screen has
the stage. The caption is the browser's, apostrophe and all.
The statistics now count screen frames apart from camera frames, by
mid, so a screen can never stand in for a camera that stopped.
Gates: swift test (40, 16 new), make native-mac's Swift build, the
iPhone simulator build, and TestNativeMacCallsBrowser/member with a new
scenario: the browser shares, the app captions and decodes the screen
while the camera keeps arriving, the browser stops, the tile goes and
the camera continues.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
11 files changed,
+594
−42
native/README.md+12 −0native/apple/Sources/CallsInterop/Interop.swift+4 −0native/apple/Sources/CallsKit/Call.swift+49 −1native/apple/Sources/CallsKit/MacCallView.swift+57 −15native/apple/Sources/CallsKit/MacVideoTile.swift+28 −9native/apple/Sources/CallsKit/Media.swift+123 −5native/apple/Sources/CallsKit/ScreenShare.swift+120 −0native/apple/Sources/CallsKit/Wire.swift+2 −0native/apple/Tests/CallsKitTests/ScreenShareTests.swift+121 −0native/ios/Sources/CallsPhone.swift+48 −12test/browser/native.mjs+30 −0
diff --git a/native/README.md b/native/README.md| index 5f6a204..fb18de2 100644 |
| --- a/native/README.md |
| +++ b/native/README.md |
| @@ -66,6 +66,18 @@ sender's clock — and a stretched voice is what a growl is. On macOS it is |
| the only instrument there is, since a bad device cannot be ruled out by |
| switching away from it. |
| +**Shared screens** from anyone in the call — a browser or the other app — |
| +take the stage as a tile of their own, captioned "Name’s screen" exactly as |
| +the browser captions it, while the sharer's camera moves to a thumbnail |
| +beside yours. Nothing on the wire says which m-line is a screen; the app |
| +works it out with the browser's own rules (spec §5.4, `classifyTrack` in |
| +`room.js`, ported in `ScreenShare.swift` with its tests), so the two never |
| +disagree about whether they are looking at a face or a document. The tile |
| +goes on whichever of §5.4's two endings arrives first: the `share {on:false}` |
| +message, or the stopped transceiver taking its receiver away. Each was |
| +checked alone against a browser when this was written (the other one |
| +disabled); the native gate checks them together. |
| + |
| **Knocks** are announced when someone asks to be let in while the app is |
| not frontmost. The admission queue is already on screen for a member who is |
| looking at the call; this is for the member who is in another Space, |
diff --git a/native/apple/Sources/CallsInterop/Interop.swift b/native/apple/Sources/CallsInterop/Interop.swift| index c358db5..84a3e92 100644 |
| --- a/native/apple/Sources/CallsInterop/Interop.swift |
| +++ b/native/apple/Sources/CallsInterop/Interop.swift |
| @@ -79,6 +79,10 @@ struct Input: Decodable { |
| "receivedChatProbe": call.chat.contains { $0.hasSuffix(": browser chat probe") }, |
| "sentChatProbe": call.chat.contains("You: native chat probe"), |
| "receivedChatCount": call.receivedChatCount, |
| + // §5.4: what the stage would caption each screen, in the order |
| + // they started. The words, not a count: the caption is the only |
| + // place a screen says whose it is. |
| + "screenCaptions": call.screens.map { screenCaption(call.name(of: $0)) }, |
| ]), let text = String(data: data, encoding: .utf8) { |
| print(text) |
| fflush(stdout) |
diff --git a/native/apple/Sources/CallsKit/Call.swift b/native/apple/Sources/CallsKit/Call.swift| index 9235300..e807ba7 100644 |
| --- a/native/apple/Sources/CallsKit/Call.swift |
| +++ b/native/apple/Sources/CallsKit/Call.swift |
| @@ -32,6 +32,12 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| @Published public private(set) var standing = "" |
| @Published public private(set) var status = "Ready to join" |
| @Published public private(set) var remoteTracks: [String: RTCVideoTrack] = [:] |
| + /// §5.4: the peers showing a screen here, oldest first, and the screens |
| + /// that have arrived. Two things and not one, because the `share` |
| + /// message usually beats the media by a renegotiation, and the tile |
| + /// carries its caption before there is anything to show in it. |
| + @Published public private(set) var screens: [String] = [] |
| + @Published public private(set) var remoteScreens: [String: RTCVideoTrack] = [:] |
| @Published public private(set) var localTrack: RTCVideoTrack? |
| @Published public private(set) var muted = false |
| @Published public private(set) var cameraOff = false |
| @@ -114,6 +120,8 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| events = nil |
| media.close() |
| remoteTracks = [:] |
| + screens = [] |
| + remoteScreens = [:] |
| localTrack = nil |
| ice = nil |
| state = nil |
| @@ -288,12 +296,16 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| ][state.reason ?? ""] ?? "The call has ended" |
| media.close() |
| remoteTracks = [:] |
| + screens = [] |
| + remoteScreens = [:] |
| localTrack = nil |
| default: status = "Connecting…" |
| } |
| if state.peer.isEmpty { |
| media.closePeers() |
| remoteTracks = [:] |
| + screens = [] |
| + remoteScreens = [:] |
| ice = nil |
| } |
| if let ice { media.reconcile(state: state, ice: ice) } |
| @@ -314,6 +326,10 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| appendChat("\(message.name ?? "Participant"): \(text)") |
| receivedChatCount += 1 |
| } |
| + if message.type == "share" { |
| + media.receiveShare(message) |
| + return |
| + } |
| if ice == nil, ["offer", "answer", "candidate"].contains(message.type) { |
| if pendingMedia.count < 256 { pendingMedia.append(message) } |
| } else { |
| @@ -327,12 +343,44 @@ private final class Events: NSObject, BridgeListenerProtocol { |
| if chat.count > 200 { chat.removeFirst(chat.count - 200) } |
| } |
| func setRemote(_ track: RTCVideoTrack, peer: String) { remoteTracks[peer] = track } |
| - func removeRemote(_ peer: String) { remoteTracks.removeValue(forKey: peer) } |
| + func removeRemote(_ peer: String) { |
| + remoteTracks.removeValue(forKey: peer) |
| + dropScreen(peer) |
| + } |
| + |
| + // --------------------------------------------------------------- |
| + // Screens (§5.4) |
| + // --------------------------------------------------------------- |
| + |
| + /// The name the roster gives a peer, which is the only place a name |
| + /// comes from: the hub put it there, and nothing a peer sends can. |
| + public func name(of peer: String) -> String { |
| + state?.peers.first { $0.peer == peer }?.name ?? "Participant" |
| + } |
| + |
| + /// A peer has said it is sharing, or its screen has arrived. Either one |
| + /// is enough to draw the tile. |
| + func announceScreen(_ peer: String) { |
| + if !screens.contains(peer) { screens.append(peer) } |
| + } |
| + func setScreen(_ track: RTCVideoTrack, peer: String) { |
| + announceScreen(peer) |
| + remoteScreens[peer] = track |
| + } |
| + /// Both of §5.4's endings meet here — the `share {on:false}` message |
| + /// and the receiver going away with its stopped transceiver — and |
| + /// whichever comes first wins. The second finds nothing to do. |
| + func dropScreen(_ peer: String) { |
| + screens.removeAll { $0 == peer } |
| + remoteScreens.removeValue(forKey: peer) |
| + } |
| func mediaFailed() { status = "A participant’s media could not connect." } |
| public func sampleStats() async -> [String: UInt64] { |
| var values = await media.stats() |
| values["hasICE"] = ice == nil ? 0 : 1 |
| values["remoteTracks"] = UInt64(remoteTracks.count) |
| + values["remoteScreens"] = UInt64(remoteScreens.count) |
| + values["screens"] = UInt64(screens.count) |
| values["invalidEvents"] = invalidEvents |
| values["rosterPeers"] = UInt64(state?.peers.count ?? 0) |
| return values |
diff --git a/native/apple/Sources/CallsKit/MacCallView.swift b/native/apple/Sources/CallsKit/MacCallView.swift| index 93a201c..4389adc 100644 |
| --- a/native/apple/Sources/CallsKit/MacCallView.swift |
| +++ b/native/apple/Sources/CallsKit/MacCallView.swift |
| @@ -6,7 +6,7 @@ |
| private let leave: () -> Void |
| @State private var grid: Bool |
| @State private var showingChat: Bool |
| - @State private var selectedPeer: String? |
| + @State private var chosen: StageItem? |
| @State private var readChatCount = 0 |
| @State private var showingDevices = false |
| @StateObject private var chatWindow = MacChatWindow() |
| @@ -29,15 +29,18 @@ |
| .accessibilityAddTraits(.updatesFrequently) |
| } |
| Spacer() |
| - if !grid, peers.count > 1 { |
| + if !grid, peers.count > 1 || !call.screens.isEmpty { |
| Menu { |
| + ForEach(call.screens, id: \.self) { peer in |
| + Button(screenCaption(call.name(of: peer))) { chosen = .screen(peer) } |
| + } |
| ForEach(peers) { peer in |
| - Button(peer.name) { selectedPeer = peer.peer } |
| + Button(peer.name) { chosen = .camera(peer.peer) } |
| } |
| } label: { |
| Label("Choose participant", systemImage: "person.2") |
| } |
| - .help("Choose the participant to show on the main stage") |
| + .help("Choose the participant or screen to show on the main stage") |
| } |
| Picker("Video layout", selection: $grid) { |
| Label("Focus", systemImage: "rectangle.inset.bottomtrailing").tag(false) |
| @@ -98,6 +101,11 @@ |
| .onChange(of: call.receivedChatCount) { _, count in |
| if showingChat || chatWindow.isOpen { readChatCount = count } |
| } |
| + .onChange(of: call.screens) { before, after in |
| + // Somebody starting to share is asking to be looked at, so a new |
| + // screen takes the stage from whatever was chosen before it. |
| + if after.contains(where: { !before.contains($0) }) { chosen = nil } |
| + } |
| .onChange(of: showingChat) { _, visible in |
| if visible { readChatCount = call.receivedChatCount } |
| } |
| @@ -110,12 +118,25 @@ |
| private var peers: [RosterPeer] { |
| call.state?.peers.filter { $0.peer != call.state?.peer } ?? [] |
| } |
| - private var focusedPeer: RosterPeer? { peers.first { $0.peer == selectedPeer } ?? peers.first } |
| + private var focused: StageItem? { |
| + Stage.focus(chosen: chosen, peers: peers.map(\.peer), screens: call.screens) |
| + } |
| + private var stageShowsScreen: Bool { |
| + if case .screen = focused { return true } |
| + return false |
| + } |
| @ViewBuilder private var stage: some View { |
| if grid { |
| ScrollView { |
| LazyVGrid(columns: [GridItem(.adaptive(minimum: 240))], spacing: 12) { |
| + ForEach(call.screens, id: \.self) { peer in |
| + MacVideoTile( |
| + track: call.remoteScreens[peer], label: screenCaption(call.name(of: peer)), |
| + screen: true |
| + ) |
| + .aspectRatio(16 / 9, contentMode: .fit) |
| + } |
| MacVideoTile( |
| track: call.localTrack, label: "You", mirrored: true, cameraOff: call.cameraOff |
| ) |
| @@ -129,9 +150,14 @@ |
| } else { |
| GeometryReader { geometry in |
| ZStack(alignment: .bottomTrailing) { |
| - if let peer = focusedPeer { |
| - MacVideoTile(track: call.remoteTracks[peer.peer], label: peer.name) |
| - } else { |
| + switch focused { |
| + case .screen(let peer): |
| + MacVideoTile( |
| + track: call.remoteScreens[peer], label: screenCaption(call.name(of: peer)), |
| + screen: true) |
| + case .camera(let peer): |
| + MacVideoTile(track: call.remoteTracks[peer], label: call.name(of: peer)) |
| + case nil: |
| ContentUnavailableView( |
| "Waiting for others", systemImage: "person.2", |
| description: Text("Your meeting is ready. Others will appear here when they join.") |
| @@ -139,19 +165,35 @@ |
| .frame(maxWidth: .infinity, maxHeight: .infinity) |
| .background(CallsAppearance.background, in: RoundedRectangle(cornerRadius: 16)) |
| } |
| - MacVideoTile( |
| - track: call.localTrack, label: "You", mirrored: true, cameraOff: call.cameraOff |
| - ) |
| - .frame( |
| - width: min(180, geometry.size.width * 0.3), |
| - height: min(135, geometry.size.height * 0.3) |
| - ) |
| + // With a screen on the stage every camera would otherwise be |
| + // off it, so the people come down to thumbnails beside yours. |
| + let others = stageShowsScreen ? peers : [] |
| + let size = thumbnail(count: others.count + 1, in: geometry.size) |
| + VStack(spacing: 10) { |
| + ForEach(others) { peer in |
| + MacVideoTile(track: call.remoteTracks[peer.peer], label: peer.name) |
| + .frame(width: size.width, height: size.height) |
| + } |
| + MacVideoTile( |
| + track: call.localTrack, label: "You", mirrored: true, cameraOff: call.cameraOff |
| + ) |
| + .frame(width: size.width, height: size.height) |
| + } |
| .shadow(color: .black.opacity(0.25), radius: 8, y: 3).padding(14) |
| } |
| } |
| } |
| } |
| + /// A thumbnail's size, when `count` of them stack up the stage's |
| + /// trailing edge: the size one always had, until there are enough of |
| + /// them that it would run off the top. |
| + private func thumbnail(count: Int, in stage: CGSize) -> CGSize { |
| + let room = (stage.height - 28 - CGFloat(count - 1) * 10) / CGFloat(max(count, 1)) |
| + let height = max(60, min(135, stage.height * 0.3, room)) |
| + return CGSize(width: min(180, stage.width * 0.3, height * 4 / 3), height: height) |
| + } |
| + |
| private var controls: some View { |
| HStack(spacing: 12) { |
| Button( |
diff --git a/native/apple/Sources/CallsKit/MacVideoTile.swift b/native/apple/Sources/CallsKit/MacVideoTile.swift| index 061b5de..8b5d73d 100644 |
| --- a/native/apple/Sources/CallsKit/MacVideoTile.swift |
| +++ b/native/apple/Sources/CallsKit/MacVideoTile.swift |
| @@ -8,17 +8,24 @@ |
| let label: String |
| var mirrored = false |
| var cameraOff = false |
| + /// A shared screen (§5.4). Shown whole rather than cropped to fill: |
| + /// the edge of a screen is where its menu bar and its scroll bars are, |
| + /// and a face survives losing its margins where a document does not. |
| + var screen = false |
| var body: some View { |
| ZStack { |
| Color.black |
| if let track, !cameraOff { |
| - MacVideoView(track: track).scaleEffect(x: mirrored ? -1 : 1, y: 1) |
| + MacVideoView(track: track, fit: screen).scaleEffect(x: mirrored ? -1 : 1, y: 1) |
| } else { |
| VStack(spacing: 12) { |
| - Image(systemName: cameraOff ? "video.slash" : "person.crop.circle") |
| - .font(.largeTitle) |
| - Text(cameraOff ? "Camera off" : "Waiting for video").font(.callout) |
| + Image( |
| + systemName: cameraOff |
| + ? "video.slash" : screen ? "rectangle.on.rectangle" : "person.crop.circle" |
| + ) |
| + .font(.largeTitle) |
| + Text(waiting).font(.callout) |
| }.foregroundStyle(.white.opacity(0.8)) |
| } |
| } |
| @@ -30,23 +37,33 @@ |
| .clipShape(RoundedRectangle(cornerRadius: 16)) |
| .accessibilityElement(children: .ignore) |
| .accessibilityLabel(label) |
| - .accessibilityValue( |
| - cameraOff ? "Camera off" : track == nil ? "Waiting for video" : "Video on") |
| + .accessibilityValue(cameraOff ? "Camera off" : track == nil ? waiting : "Video on") |
| + } |
| + private var waiting: String { |
| + cameraOff ? "Camera off" : screen ? "Waiting for the screen" : "Waiting for video" |
| } |
| } |
| private struct MacVideoView: NSViewRepresentable { |
| let track: RTCVideoTrack |
| + var fit = false |
| func makeNSView(context: Context) -> FillVideoView { FillVideoView() } |
| - func updateNSView(_ view: FillVideoView, context: Context) { view.attach(track) } |
| + func updateNSView(_ view: FillVideoView, context: Context) { |
| + view.fit = fit |
| + view.attach(track) |
| + } |
| static func dismantleNSView(_ view: FillVideoView, coordinator: ()) { view.detach() } |
| } |
| - /// The renderer keeps its source aspect ratio while the container crops overflow. |
| + /// The renderer keeps its source aspect ratio while the container crops |
| + /// overflow — or, for a screen, letterboxes it. |
| private final class FillVideoView: NSView, RTCVideoViewDelegate { |
| private let renderer = RTCMTLNSVideoView(frame: .zero) |
| private var track: RTCVideoTrack? |
| private var videoSize = NSSize(width: 16, height: 9) |
| + var fit = false { |
| + didSet { if fit != oldValue { needsLayout = true } } |
| + } |
| override init(frame: NSRect) { |
| super.init(frame: frame) |
| @@ -78,7 +95,9 @@ |
| override func layout() { |
| super.layout() |
| - let scale = max(bounds.width / videoSize.width, bounds.height / videoSize.height) |
| + let across = bounds.width / videoSize.width |
| + let down = bounds.height / videoSize.height |
| + let scale = fit ? min(across, down) : max(across, down) |
| let size = NSSize(width: videoSize.width * scale, height: videoSize.height * scale) |
| renderer.frame = NSRect( |
| x: (bounds.width - size.width) / 2, |
diff --git a/native/apple/Sources/CallsKit/Media.swift b/native/apple/Sources/CallsKit/Media.swift| index 46e73ca..15d358a 100644 |
| --- a/native/apple/Sources/CallsKit/Media.swift |
| +++ b/native/apple/Sources/CallsKit/Media.swift |
| @@ -89,6 +89,14 @@ import WebRTC |
| } |
| } |
| func receive(_ message: Message) { if let from = message.from { peers[from]?.receive(message) } } |
| + /// §5.4's label. A peer this app has no connection to yet is ignored, |
| + /// as the browser ignores it: its screen will be told apart from its |
| + /// camera by the m-line when the connection comes, and a caption with |
| + /// no connection behind it is a tile nothing will ever fill or clear. |
| + func receiveShare(_ message: Message) { |
| + guard let from = message.from, let peer = peers[from] else { return } |
| + peer.share(on: message.payload?.on == true) |
| + } |
| func updateICE(_ config: ICEConfig) { for p in peers.values { p.updateICE(config) } } |
| /// The five audio-health fields, gathered alongside the byte counts. |
| /// |
| @@ -117,6 +125,7 @@ import WebRTC |
| func stats() async -> [String: UInt64] { |
| var values: [String: UInt64] = [ |
| "framesDecoded": 0, "audioBytesReceived": 0, "videoBytesSent": 0, "audioBytesSent": 0, |
| + "screenFramesDecoded": 0, "screenBytesSent": 0, |
| "peerConnections": UInt64(peers.count), |
| "concealedSamples": 0, "totalSamplesReceived": 0, |
| "insertedSamplesForDeceleration": 0, "removedSamplesForAcceleration": 0, |
| @@ -129,9 +138,15 @@ import WebRTC |
| } |
| for stat in report.statistics.values { |
| let kind = (stat.values["kind"] as? String) ?? (stat.values["mediaType"] as? String) ?? "" |
| + let mid = stat.values["mid"] as? String ?? "" |
| if stat.type == "inbound-rtp" { |
| if kind == "video" { |
| - values["framesDecoded", default: 0] += |
| + // A peer sharing its screen sends two video streams, and a |
| + // count that added them together would let a screen stand in |
| + // for a camera that had stopped. Told apart by mid, which is |
| + // what the browser's own gate does. |
| + let field = peer.isScreen(mid: mid) ? "screenFramesDecoded" : "framesDecoded" |
| + values[field, default: 0] += |
| (stat.values["framesDecoded"] as? NSNumber)?.uint64Value ?? 0 |
| } |
| if kind == "audio" { |
| @@ -151,7 +166,8 @@ import WebRTC |
| } |
| if stat.type == "outbound-rtp" { |
| if kind == "video" { |
| - values["videoBytesSent", default: 0] += |
| + let field = peer.isOwnScreen(mid: mid) ? "screenBytesSent" : "videoBytesSent" |
| + values[field, default: 0] += |
| (stat.values["bytesSent"] as? NSNumber)?.uint64Value ?? 0 |
| } |
| if kind == "audio" { |
| @@ -165,6 +181,19 @@ import WebRTC |
| } |
| } |
| +/// A transceiver direction as SDP spells it, which is how the classifier, |
| +/// and the browser it was ported from, compare them. |
| +private func sdpDirection(_ direction: RTCRtpTransceiverDirection) -> String { |
| + switch direction { |
| + case .sendRecv: return "sendrecv" |
| + case .sendOnly: return "sendonly" |
| + case .recvOnly: return "recvonly" |
| + case .inactive: return "inactive" |
| + case .stopped: return "stopped" |
| + @unknown default: return "" |
| + } |
| +} |
| + |
| @MainActor private final class MediaPeer: NSObject, RTCPeerConnectionDelegate { |
| let id: String |
| let generation: Int64 |
| @@ -177,6 +206,17 @@ import WebRTC |
| private var ignoreOffer = false |
| private var candidates: [Candidate] = [] |
| private var lastICE: ICEConfig? |
| + // §5.4, receiving. What this peer's camera and microphone came in on, |
| + // and whether it has said it is sharing; the receiver its screen |
| + // arrived on, so that only *that* receiver going away takes the tile |
| + // with it; and the mid, so the statistics can count the two video |
| + // streams apart. |
| + private var clues = ScreenClues() |
| + private var screenReceiver: String? |
| + private var screenMid = "" |
| + // Whether this connection carries a camera of ours: the classifier's |
| + // direction rule turns on it. |
| + private var sentVideo = false |
| init(id: String, generation: Int64, polite: Bool, engine: MediaEngine) { |
| self.id = id |
| @@ -201,8 +241,56 @@ import WebRTC |
| constraints: RTCMediaConstraints(mandatoryConstraints: nil, optionalConstraints: nil), |
| delegate: self) |
| if let audio = engine.localAudio { _ = pc?.add(audio, streamIds: ["calls"]) } |
| - if let video = engine.localVideo { _ = pc?.add(video, streamIds: ["calls"]) } |
| + if let video = engine.localVideo { |
| + sentVideo = pc?.add(video, streamIds: ["calls"]) != nil |
| + } |
| + } |
| + func isScreen(mid: String) -> Bool { !mid.isEmpty && mid == screenMid } |
| + func isOwnScreen(mid: String) -> Bool { false } |
| + |
| + /// The `share` message: on draws the captioned tile ahead of the media, |
| + /// off is one of §5.4's two endings. |
| + func share(on: Bool) { |
| + if on { |
| + clues.sharing = true |
| + engine?.call?.announceScreen(id) |
| + } else { |
| + dropScreen() |
| + } |
| + } |
| + private func dropScreen() { |
| + clues.sharing = false |
| + screenReceiver = nil |
| + screenMid = "" |
| + engine?.call?.dropScreen(id) |
| + } |
| + |
| + /// One track arriving, put in its place. A camera goes on the peer's |
| + /// tile exactly as it always did; a screen gets a tile of its own and |
| + /// leaves the camera alone. |
| + private func arrived( |
| + _ track: RTCMediaStreamTrack, receiver: String, info: ArrivingTrack |
| + ) { |
| + let role = classifyTrack(clues, info, hasVideo: sentVideo) |
| + if role == .screen, let video = track as? RTCVideoTrack { |
| + clues.sharing = true |
| + screenReceiver = receiver |
| + if !info.mid.isEmpty { screenMid = info.mid } |
| + engine?.call?.setScreen(video, peer: id) |
| + return |
| + } |
| + if role == .camera { |
| + if !info.mid.isEmpty { clues.cameraMid = info.mid } |
| + if !info.streamId.isEmpty { clues.cameraStream = info.streamId } |
| + } |
| + // Recorded from the first track of any kind: it is what places a |
| + // video track for a peer that has no camera. |
| + if clues.peerStream.isEmpty { clues.peerStream = info.streamId } |
| + if role == .camera, let video = track as? RTCVideoTrack { |
| + engine?.call?.setRemote(video, peer: id) |
| + } |
| } |
| + |
| func close() { |
| closed = true |
| work?.cancel() |
| @@ -355,10 +443,40 @@ import WebRTC |
| if let audio = rtpReceiver.track as? RTCAudioTrack, !playAudio { |
| audio.source.volume = 0 |
| } |
| - guard let track = rtpReceiver.track as? RTCVideoTrack else { return } |
| + guard let track = rtpReceiver.track else { return } |
| + // Read here, on the signalling thread this callback arrives on, rather |
| + // than on the main actor: by the time the main actor gets to it the |
| + // next description can have moved the transceiver on. |
| + let receiver = rtpReceiver.receiverId |
| + let transceiver = peerConnection.transceivers.first { $0.receiver.receiverId == receiver } |
| + var info = ArrivingTrack(kind: track.kind, streamId: streams.first?.streamId ?? "") |
| + if let transceiver { |
| + info.mid = transceiver.mid |
| + // The negotiated direction when there is one. On the side that |
| + // answers an offer there is not yet, and that is exactly the side a |
| + // screen arrives at; the direction the transceiver was built with is |
| + // recvonly for an m-line the far side added, which is the point. |
| + var current = RTCRtpTransceiverDirection.inactive |
| + info.direction = sdpDirection( |
| + transceiver.currentDirection(¤t) ? current : transceiver.direction) |
| + } |
| Task { @MainActor [weak self] in |
| guard let self, !self.closed else { return } |
| - self.engine?.call?.setRemote(track, peer: self.id) |
| + self.arrived(track, receiver: receiver, info: info) |
| + } |
| + } |
| + /// §5.4's other ending: the sender stopped its screen's transceiver and |
| + /// the renegotiation took the receiver away. Only the receiver the |
| + /// screen is showing from counts. A screen replaced on its own mid ends |
| + /// its predecessor afterwards, and without the check that took the live |
| + /// replacement's tile with it (the browser's Codex S4-15). |
| + nonisolated func peerConnection( |
| + _ peerConnection: RTCPeerConnection, didRemove rtpReceiver: RTCRtpReceiver |
| + ) { |
| + let receiver = rtpReceiver.receiverId |
| + Task { @MainActor [weak self] in |
| + guard let self, !self.closed, self.screenReceiver == receiver else { return } |
| + self.dropScreen() |
| } |
| } |
| nonisolated func peerConnection( |
diff --git a/native/apple/Sources/CallsKit/ScreenShare.swift b/native/apple/Sources/CallsKit/ScreenShare.swift| new file mode 100644 |
| index 0000000..c3661f1 |
| --- /dev/null |
| +++ b/native/apple/Sources/CallsKit/ScreenShare.swift |
| @@ -0,0 +1,120 @@ |
| +import Foundation |
| + |
| +// --------------------------------------------------------------- |
| +// Screen share, receiving (spec §5.4) |
| +// --------------------------------------------------------------- |
| +// |
| +// The web client is the contract here, and this file is its receiving |
| +// half ported rule for rule: a sharer adds its display as one `sendonly` |
| +// transceiver of its own, in a stream of its own, announces it with a |
| +// `share {on:true}` broadcast, and stops it by stopping the transceiver |
| +// and broadcasting `share {on:false}`. Nothing on the wire says which |
| +// m-line is the screen. Each receiver works it out, and a native app and |
| +// a browser looking at the same sender have to reach the same answer, or |
| +// one of them captions a face as a screen. |
| + |
| +/// What one arriving track is to the person looking at it. |
| +enum TrackRole: Equatable { |
| + case audio, camera, screen |
| +} |
| + |
| +/// What a connection has recorded about the peer on the other end: the |
| +/// mid and stream its camera came in on, the stream everything else from |
| +/// it has arrived in, and whether it has said it is sharing. room.js's |
| +/// `entry` fields of the same names. |
| +struct ScreenClues: Equatable { |
| + var cameraMid = "" |
| + var cameraStream = "" |
| + var peerStream = "" |
| + var sharing = false |
| +} |
| + |
| +/// One arriving track, as the classifier needs it. `direction` is the |
| +/// transceiver's negotiated direction when there is one and the direction |
| +/// it was built with when there is not, spelled the way SDP spells it. |
| +struct ArrivingTrack: Equatable { |
| + var kind: String |
| + var streamId = "" |
| + var mid = "" |
| + var direction = "" |
| +} |
| + |
| +/// Tells a peer's screen from its camera. room.js's classifyTrack, in |
| +/// the same order and for the same reasons, which are set out there at |
| +/// length; the short form: |
| +/// |
| +/// 1. A video track arriving `recvonly` on a connection that sends a |
| +/// camera is on an m-line the far side added, and the only one it adds |
| +/// is the screen. |
| +/// 2. The camera's own mid, once one is known. |
| +/// 3. The camera's stream id, for a sender that reports no mid. |
| +/// 4. The stream the peer's microphone came in: a sender puts its camera |
| +/// there and its screen somewhere else. |
| +/// |
| +/// And when none of those can speak, the `share` message breaks the tie. |
| +/// It is last on purpose: it says the peer is showing a screen somewhere, |
| +/// not that this track is it. |
| +/// |
| +/// `hasVideo` is whether this connection carries a camera of ours. The |
| +/// apps always add one, so rule 1 is the rule that decides in practice; |
| +/// the rest are here because a browser on the far side may not have one, |
| +/// and a port that dropped them would disagree with the browser the day |
| +/// that mattered. |
| +func classifyTrack(_ known: ScreenClues, _ track: ArrivingTrack, hasVideo: Bool) -> TrackRole { |
| + if track.kind != "video" { return .audio } |
| + if track.direction == "recvonly" && hasVideo { return .screen } |
| + if !known.cameraMid.isEmpty && !track.mid.isEmpty { |
| + return track.mid == known.cameraMid ? .camera : .screen |
| + } |
| + if !known.cameraStream.isEmpty && !track.streamId.isEmpty { |
| + return track.streamId == known.cameraStream ? .camera : .screen |
| + } |
| + if !known.peerStream.isEmpty && !track.streamId.isEmpty { |
| + return track.streamId == known.peerStream ? .camera : .screen |
| + } |
| + if known.cameraMid.isEmpty && known.cameraStream.isEmpty { |
| + return known.sharing ? .screen : .camera |
| + } |
| + // A camera is known but there is nothing on this track to compare with |
| + // it: a second video track is the screen. |
| + return .screen |
| +} |
| + |
| +/// The words under a screen. The browser's, character for character — |
| +/// including the typographic apostrophe — so a person moving between the |
| +/// app and a browser tab reads the same caption on the same screen. |
| +public func screenCaption(_ name: String) -> String { name + "’s screen" } |
| + |
| +/// One thing the main stage can show: a person's camera or their screen. |
| +public enum StageItem: Hashable, Sendable { |
| + case camera(String) |
| + case screen(String) |
| + |
| + public var peer: String { |
| + switch self { |
| + case .camera(let peer), .screen(let peer): return peer |
| + } |
| + } |
| +} |
| + |
| +public enum Stage { |
| + /// What the main stage shows. |
| + /// |
| + /// A choice the person made stands for as long as it is still there to |
| + /// show. Otherwise a screen beats a camera, and the newest screen beats |
| + /// an older one: somebody who starts sharing is asking to be looked at, |
| + /// and a call with a screen on it is usually a call about the screen. |
| + /// With no screens it is the first camera, which is where the stage |
| + /// always was. |
| + public static func focus(chosen: StageItem?, peers: [String], screens: [String]) -> StageItem? { |
| + if let chosen { |
| + switch chosen { |
| + case .camera(let peer) where peers.contains(peer): return chosen |
| + case .screen(let peer) where screens.contains(peer): return chosen |
| + default: break |
| + } |
| + } |
| + if let newest = screens.last { return .screen(newest) } |
| + return peers.first.map(StageItem.camera) |
| + } |
| +} |
diff --git a/native/apple/Sources/CallsKit/Wire.swift b/native/apple/Sources/CallsKit/Wire.swift| index 6abe484..c1c3109 100644 |
| --- a/native/apple/Sources/CallsKit/Wire.swift |
| +++ b/native/apple/Sources/CallsKit/Wire.swift |
| @@ -36,6 +36,8 @@ struct Payload: Decodable { |
| let description: Description? |
| let candidate: Candidate? |
| let text: String? |
| + /// `share`'s one field (§5.4): whether the sender is showing a screen. |
| + let on: Bool? |
| } |
| struct Description: Codable { |
| let type: String |
diff --git a/native/apple/Tests/CallsKitTests/ScreenShareTests.swift b/native/apple/Tests/CallsKitTests/ScreenShareTests.swift| new file mode 100644 |
| index 0000000..bc2d784 |
| --- /dev/null |
| +++ b/native/apple/Tests/CallsKitTests/ScreenShareTests.swift |
| @@ -0,0 +1,121 @@ |
| +import XCTest |
| + |
| +@testable import CallsKit |
| + |
| +/// §5.4's receiving half. These are room-share.test.mjs's classifyTrack |
| +/// cases, case for case: the browser and the apps look at the same |
| +/// senders, and the day they disagree about which m-line is a screen, one |
| +/// of them captions a face as a screen. A case added there belongs here. |
| +final class ScreenShareTests: XCTestCase { |
| + private let nothing = ScreenClues() |
| + private let known = ScreenClues(cameraMid: "0", cameraStream: "cam", peerStream: "cam") |
| + private func video(_ stream: String = "", mid: String = "", direction: String = "") |
| + -> ArrivingTrack |
| + { |
| + ArrivingTrack(kind: "video", streamId: stream, mid: mid, direction: direction) |
| + } |
| + |
| + func testAudioIsNeverAScreen() { |
| + let audio = ArrivingTrack(kind: "audio", streamId: "other", mid: "9") |
| + XCTAssertEqual(classifyTrack(nothing, audio, hasVideo: true), .audio) |
| + XCTAssertEqual(classifyTrack(known, audio, hasVideo: true), .audio) |
| + } |
| + |
| + func testTheFirstVideoTrackIsTheCamera() { |
| + XCTAssertEqual(classifyTrack(nothing, video("cam", mid: "0"), hasVideo: false), .camera) |
| + XCTAssertEqual(classifyTrack(nothing, video(), hasVideo: false), .camera) |
| + } |
| + |
| + func testASecondVideoTrackOnAnotherTransceiverIsTheScreen() { |
| + XCTAssertEqual(classifyTrack(known, video("screen", mid: "2"), hasVideo: false), .screen) |
| + } |
| + |
| + func testTheCameraAgainIsStillTheCamera() { |
| + XCTAssertEqual(classifyTrack(known, video("cam", mid: "0"), hasVideo: false), .camera) |
| + // The mid decides when there is one, whatever the stream is now called. |
| + XCTAssertEqual(classifyTrack(known, video("renamed", mid: "0"), hasVideo: false), .camera) |
| + } |
| + |
| + func testWithNoMidTheStreamDecides() { |
| + let byStream = ScreenClues(cameraStream: "cam") |
| + XCTAssertEqual(classifyTrack(byStream, video("cam"), hasVideo: false), .camera) |
| + XCTAssertEqual(classifyTrack(byStream, video("screen"), hasVideo: false), .screen) |
| + XCTAssertEqual(classifyTrack(known, video("cam"), hasVideo: false), .camera) |
| + XCTAssertEqual(classifyTrack(known, video("screen"), hasVideo: false), .screen) |
| + } |
| + |
| + func testASecondVideoTrackWithNothingToCompareIsTheScreen() { |
| + XCTAssertEqual(classifyTrack(known, video(), hasVideo: false), .screen) |
| + } |
| + |
| + func testRecvonlyIsTheScreenWhenThisSideSendsACamera() { |
| + // The rule that decides in practice here: the apps always add a |
| + // camera, so a video m-line arriving recvonly is one the far side |
| + // added, and a screen is the only one it adds — even as the first |
| + // video track that peer ever sends. |
| + XCTAssertEqual( |
| + classifyTrack(nothing, video("screen", mid: "2", direction: "recvonly"), hasVideo: true), |
| + .screen) |
| + } |
| + |
| + func testRecvonlySaysNothingWithoutACameraOfOurOwn() { |
| + XCTAssertEqual( |
| + classifyTrack(nothing, video("cam", mid: "1", direction: "recvonly"), hasVideo: false), |
| + .camera) |
| + } |
| + |
| + func testASendrecvCameraIsACamera() { |
| + XCTAssertEqual( |
| + classifyTrack(nothing, video("cam", mid: "1", direction: "sendrecv"), hasVideo: true), |
| + .camera) |
| + } |
| + |
| + func testAVideoTrackOutsideTheMicrophonesStreamIsTheScreen() { |
| + let audioOnly = ScreenClues(peerStream: "theirs") |
| + XCTAssertEqual(classifyTrack(audioOnly, video("screen", mid: "2"), hasVideo: false), .screen) |
| + XCTAssertEqual(classifyTrack(audioOnly, video("theirs", mid: "1"), hasVideo: false), .camera) |
| + } |
| + |
| + func testTheShareMessageBreaksTheTieAndOnlyTheTie() { |
| + XCTAssertEqual(classifyTrack(ScreenClues(sharing: true), video(), hasVideo: false), .screen) |
| + XCTAssertEqual(classifyTrack(ScreenClues(sharing: false), video(), hasVideo: false), .camera) |
| + let sharing = ScreenClues(peerStream: "theirs", sharing: true) |
| + XCTAssertEqual(classifyTrack(sharing, video("theirs", mid: "1"), hasVideo: false), .camera) |
| + XCTAssertEqual( |
| + classifyTrack(ScreenClues(cameraMid: "1", sharing: true), video(mid: "1"), hasVideo: false), |
| + .camera) |
| + } |
| + |
| + func testTheCaptionIsTheBrowsers() { |
| + // Typographic apostrophe, as room.js's shareCaption writes it. |
| + XCTAssertEqual(screenCaption("Bea"), "Bea\u{2019}s screen") |
| + } |
| + |
| + // --------------------------------------------------------------- |
| + // The stage |
| + // --------------------------------------------------------------- |
| + |
| + func testTheStageIsTheFirstCameraWithNoScreens() { |
| + XCTAssertEqual(Stage.focus(chosen: nil, peers: ["a", "b"], screens: []), .camera("a")) |
| + XCTAssertNil(Stage.focus(chosen: nil, peers: [], screens: [])) |
| + } |
| + |
| + func testTheNewestScreenTakesTheStage() { |
| + XCTAssertEqual(Stage.focus(chosen: nil, peers: ["a", "b"], screens: ["b"]), .screen("b")) |
| + XCTAssertEqual(Stage.focus(chosen: nil, peers: ["a", "b"], screens: ["b", "a"]), .screen("a")) |
| + } |
| + |
| + func testAChoiceStandsWhileItIsThere() { |
| + XCTAssertEqual( |
| + Stage.focus(chosen: .camera("a"), peers: ["a", "b"], screens: ["b"]), .camera("a")) |
| + XCTAssertEqual( |
| + Stage.focus(chosen: .screen("a"), peers: ["a", "b"], screens: ["a", "b"]), .screen("a")) |
| + } |
| + |
| + func testAChoiceThatWentAwayFallsBack() { |
| + // A stopped screen, and a person who left: neither leaves the stage |
| + // empty while there is something else to show. |
| + XCTAssertEqual(Stage.focus(chosen: .screen("a"), peers: ["a", "b"], screens: []), .camera("a")) |
| + XCTAssertEqual(Stage.focus(chosen: .camera("c"), peers: ["a"], screens: ["a"]), .screen("a")) |
| + } |
| +} |
diff --git a/native/ios/Sources/CallsPhone.swift b/native/ios/Sources/CallsPhone.swift| index 6bad8fd..7d8cef6 100644 |
| --- a/native/ios/Sources/CallsPhone.swift |
| +++ b/native/ios/Sources/CallsPhone.swift |
| @@ -66,11 +66,13 @@ struct PhoneCall: View { |
| @State private var showingDevices = false |
| @State private var readChatCount = 0 |
| @State private var grid = false |
| - @State private var selectedPeer: String? |
| + @State private var chosen: StageItem? |
| @StateObject private var draft = ChatDraft() |
| private var peers: [RosterPeer] { call.state?.peers.filter { $0.peer != call.state?.peer } ?? [] } |
| - private var focused: RosterPeer? { peers.first { $0.peer == selectedPeer } ?? peers.first } |
| + private var focused: StageItem? { |
| + Stage.focus(chosen: chosen, peers: peers.map(\.peer), screens: call.screens) |
| + } |
| var body: some View { |
| VStack(spacing: 12) { |
| @@ -85,11 +87,17 @@ struct PhoneCall: View { |
| Label("Focus", systemImage: "rectangle.inset.bottomtrailing").tag(false) |
| Label("Grid", systemImage: "square.grid.2x2").tag(true) |
| } |
| - if peers.count > 1 { |
| + if peers.count > 1 || !call.screens.isEmpty { |
| Section("Focus on") { |
| + ForEach(call.screens, id: \.self) { peer in |
| + Button(screenCaption(call.name(of: peer))) { |
| + chosen = .screen(peer) |
| + grid = false |
| + } |
| + } |
| ForEach(peers) { peer in |
| Button(peer.name) { |
| - selectedPeer = peer.peer |
| + chosen = .camera(peer.peer) |
| grid = false |
| } |
| } |
| @@ -109,6 +117,13 @@ struct PhoneCall: View { |
| if grid { |
| ScrollView { |
| LazyVGrid(columns: [GridItem(.adaptive(minimum: 240))], spacing: 12) { |
| + ForEach(call.screens, id: \.self) { peer in |
| + PhoneVideoTile( |
| + track: call.remoteScreens[peer], label: screenCaption(call.name(of: peer)), |
| + screen: true |
| + ) |
| + .aspectRatio(4 / 3, contentMode: .fit) |
| + } |
| PhoneVideoTile( |
| track: call.localTrack, label: "You", mirrored: true, cameraOff: call.cameraOff |
| ) |
| @@ -122,9 +137,14 @@ struct PhoneCall: View { |
| } else { |
| GeometryReader { geometry in |
| ZStack(alignment: .bottomTrailing) { |
| - if let peer = focused { |
| - PhoneVideoTile(track: call.remoteTracks[peer.peer], label: peer.name) |
| - } else { |
| + switch focused { |
| + case .screen(let peer): |
| + PhoneVideoTile( |
| + track: call.remoteScreens[peer], label: screenCaption(call.name(of: peer)), |
| + screen: true) |
| + case .camera(let peer): |
| + PhoneVideoTile(track: call.remoteTracks[peer], label: call.name(of: peer)) |
| + case nil: |
| ContentUnavailableView( |
| "Waiting for others", systemImage: "person.2", |
| description: Text("Others will appear here when they join.") |
| @@ -191,6 +211,10 @@ struct PhoneCall: View { |
| }.padding(16).background(.bar) |
| } |
| .task { await call.watchAudio() } |
| + .onChange(of: call.screens) { before, after in |
| + // A new screen takes the stage, as it does on the Mac. |
| + if after.contains(where: { !before.contains($0) }) { chosen = nil } |
| + } |
| .sheet(isPresented: $showingDevices) { |
| NavigationStack { |
| ScrollView { DevicePanel(call: call).frame(maxWidth: .infinity, alignment: .leading) } |
| @@ -224,14 +248,21 @@ struct PhoneVideoTile: View { |
| let label: String |
| var mirrored = false |
| var cameraOff = false |
| + /// A shared screen (§5.4), shown whole rather than cropped: on a phone |
| + /// more than anywhere, cropping a laptop's screen to a portrait tile |
| + /// would leave the middle third of it. |
| + var screen = false |
| var body: some View { |
| ZStack { |
| Color.black |
| if let track, !cameraOff { |
| - PhoneVideo(track: track).scaleEffect(x: mirrored ? -1 : 1, y: 1) |
| + PhoneVideo(track: track, fit: screen).scaleEffect(x: mirrored ? -1 : 1, y: 1) |
| } else { |
| - Image(systemName: cameraOff ? "video.slash" : "person.crop.circle") |
| - .font(.largeTitle).foregroundStyle(.white.opacity(0.8)) |
| + Image( |
| + systemName: cameraOff |
| + ? "video.slash" : screen ? "rectangle.on.rectangle" : "person.crop.circle" |
| + ) |
| + .font(.largeTitle).foregroundStyle(.white.opacity(0.8)) |
| } |
| } |
| .overlay(alignment: .bottomLeading) { |
| @@ -241,19 +272,24 @@ struct PhoneVideoTile: View { |
| } |
| .clipShape(RoundedRectangle(cornerRadius: 18)) |
| .accessibilityElement(children: .ignore).accessibilityLabel(label) |
| - .accessibilityValue(cameraOff ? "Camera off" : track == nil ? "Waiting for video" : "Video on") |
| + .accessibilityValue( |
| + cameraOff |
| + ? "Camera off" |
| + : track == nil ? (screen ? "Waiting for the screen" : "Waiting for video") : "Video on") |
| } |
| } |
| struct PhoneVideo: UIViewRepresentable { |
| let track: RTCVideoTrack |
| + var fit = false |
| func makeCoordinator() -> Coordinator { Coordinator() } |
| func makeUIView(context: Context) -> RTCMTLVideoView { |
| let view = RTCMTLVideoView(frame: .zero) |
| - view.videoContentMode = .scaleAspectFill |
| + view.videoContentMode = fit ? .scaleAspectFit : .scaleAspectFill |
| context.coordinator.attach(track, to: view) |
| return view |
| } |
| func updateUIView(_ view: RTCMTLVideoView, context: Context) { |
| + view.videoContentMode = fit ? .scaleAspectFit : .scaleAspectFill |
| context.coordinator.attach(track, to: view) |
| } |
| static func dismantleUIView(_ view: RTCMTLVideoView, coordinator: Coordinator) { |
diff --git a/test/browser/native.mjs b/test/browser/native.mjs| index e104eec..10c6507 100644 |
| --- a/test/browser/native.mjs |
| +++ b/test/browser/native.mjs |
| @@ -9,6 +9,9 @@ const browser = await chromium.launch({ headless: true, args: [ |
| '--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream', |
| '--autoplay-policy=no-user-gesture-required', '--disable-features=WebRtcHideLocalIpsWithMdns', |
| '--mute-audio', |
| + // §5.4: answers getDisplayMedia with a fake monitor that really |
| + // produces frames, as call.mjs does for the browser-only share gate. |
| + '--auto-accept-this-tab-capture', |
| ] }); |
| const context = await browser.newContext({ permissions: ['camera', 'microphone'] }); |
| await context.addCookies([{ name: process.env.CALLS_COOKIE_NAME, value: process.env.CALLS_COOKIE_A, url: origin }]); |
| @@ -125,6 +128,33 @@ try { |
| await wait(async () => await remoteFrames() > refusedFrames + 15, 'media survives a refused camera switch'); |
| console.log('PASS camera switched mid-call without disturbing the connection'); |
| } |
| + // §5.4, receiving: the browser shares its screen and the app shows it |
| + // as a tile of its own, captioned with the sharer's name, while the |
| + // browser's camera keeps arriving on the tile it always had. Then the |
| + // browser stops, and the tile goes — by the `share {on:false}` or the |
| + // stopped transceiver, whichever lands first — with the camera still |
| + // arriving afterwards. |
| + if (mode !== 'guest') { |
| + const name = await page.evaluate(() => document.getElementById('room')?.dataset.selfName || ''); |
| + assert.notEqual(name, '', 'the browser has a name for its screen to be captioned with'); |
| + const caption = name + '\u2019s screen'; |
| + await page.click('#share'); |
| + await wait(() => sample?.screenCaptions?.length === 1 && sample.screenCaptions[0] === caption, |
| + 'native captions the browser\u2019s screen with its name'); |
| + await wait(() => sample?.stats.remoteScreens === 1, 'the browser\u2019s screen reaches the app as a track'); |
| + const screenBase = sample.stats.screenFramesDecoded; |
| + await wait(() => sample.stats.screenFramesDecoded > screenBase + 15, 'native decodes the browser\u2019s screen'); |
| + const cameraBase = sample.stats.framesDecoded; |
| + await wait(() => sample.stats.framesDecoded > cameraBase + 15, 'the browser\u2019s camera keeps arriving beside its screen'); |
| + assert.equal(sample.stats.remoteTracks, 1, 'the screen is not taken for a second camera'); |
| + console.log('PASS native shows the browser\u2019s screen as its own captioned tile'); |
| + await page.click('#share'); |
| + await wait(() => sample?.screenCaptions?.length === 0 && sample.stats.remoteScreens === 0 && sample.stats.screens === 0, |
| + 'native removes the browser\u2019s screen when it stops'); |
| + const afterStop = sample.stats.framesDecoded; |
| + await wait(() => sample.stats.framesDecoded > afterStop + 15, 'the browser\u2019s camera survives its screen stopping'); |
| + console.log('PASS native removes a stopped screen and keeps the camera'); |
| + } |
| await page.locator('[data-chat-text]').fill('browser chat probe'); |
| await page.locator('[data-chat-send]').click(); |
| await wait(() => sample?.receivedChatProbe === true, 'native displays browser chat'); |