Camera sharing, and the proxy's relay for the desktop app #12

Merged
kibi merged 1 commits from screen-sharing into main 2026-10-01 21:55:00 +00:00
19 changed files with 208 additions and 78 deletions
Showing only changes of commit 819db06bc6 - Show all commits
+2
View File
@@ -6,6 +6,8 @@ All notable changes to mumh5. Versions follow the `version` in `package.json`.
### Added ### Added
- Camera sharing, as a second source in the screen sharing dialog.
- A relay for streams in the web proxy, used automatically by the browser version and, with its address entered in Settings, by the desktop app, for networks and browsers that allow no direct connection.
- Screen sharing between mumh5 users in a channel: a dialog to choose the screen or window (with preview), the sound and the quality; sound from one program or the whole system on Linux through PipeWire. Viewers watch in the voice tiles, enlarge a stream like in a meeting and set its volume. Streams go directly between clients; a STUN server can be set in Settings, Voice. - Screen sharing between mumh5 users in a channel: a dialog to choose the screen or window (with preview), the sound and the quality; sound from one program or the whole system on Linux through PipeWire. Viewers watch in the voice tiles, enlarge a stream like in a meeting and set its volume. Streams go directly between clients; a STUN server can be set in Settings, Voice.
- Browser version: `npm run build:web` builds the web app and a small self-hosted proxy that bridges browsers to Mumble servers on an allowlist. Identities are kept in the browser, voice goes through the TCP tunnel. See "Browser version" in the README. - Browser version: `npm run build:web` builds the web app and a small self-hosted proxy that bridges browsers to Mumble servers on an allowlist. Identities are kept in the browser, voice goes through the TCP tunnel. See "Browser version" in the README.
- Link previews in chat: title, description and image for web links (up to two per message). By default they are fetched by your f0ckm upload host, so the linked sites never see your IP address. Can be switched to "fetched by this computer" or off in Settings, Chat and files. - Link previews in chat: title, description and image for web links (up to two per message). By default they are fetched by your f0ckm upload host, so the linked sites never see your IP address. Can be switched to "fetched by this computer" or off in Settings, Chat and files.
+1 -1
View File
@@ -43,7 +43,7 @@ A reused test server keeps registrations and channels from earlier runs; tests m
## Architecture ## Architecture
- `electron/` (Node, main process): window, TLS sockets to Mumble servers (`tls-transport.ts`), encrypted UDP voice (`udp-voice.ts`, `ocb2.ts`, tested against Mumble's OCB2 vectors), identities and PKCS#12 (`identity.ts`, `identity-store.ts`), certificate parsing (`certs.ts`), tray (`tray.ts`). The renderer only gets the narrow `window.mumh5Native` API from `preload.ts` (context isolation, sandbox). - `electron/` (Node, main process): window, TLS sockets to Mumble servers (`tls-transport.ts`), encrypted UDP voice (`udp-voice.ts`, `ocb2.ts`, tested against Mumble's OCB2 vectors), identities and PKCS#12 (`identity.ts`, `identity-store.ts`), certificate parsing (`certs.ts`), tray (`tray.ts`). The renderer only gets the narrow `window.mumh5Native` API from `preload.ts` (context isolation, sandbox).
- `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect. `turn.ts` is its STUN and TURN server for screen sharing between browser users (UDP and TCP on one port, credentials from `/api/turn`, secret made up at start). In the browser build `localStorage mumh5.iceDebug = relay` or `relay-tcp` limits a client to relayed routes, which is how the E2E exercises the relay. - `server/` (Node): the web proxy. `proxy.ts` serves `dist-web`, bridges WebSocket connections to Mumble over TLS (reusing `electron/tls-transport.ts`) and has stateless identity endpoints. It stores nothing; the browser keeps identities in localStorage and sends one with each connect. `turn.ts` is its STUN and TURN server for screen sharing between browser users (UDP and TCP on one port, credentials from `GET /api/relay`, which any origin may read so the desktop app can use a proxy as its relay; secret made up at start). `localStorage mumh5.iceDebug = relay` or `relay-tcp` limits a client to relayed routes, which is how the E2E exercises the relay.
- `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here. - `src/core/` (browser-safe TypeScript, also runs in Node for tests): framing and codec (`proto.ts`), the Mumble client state machine (`client.ts`), voice packet formats (`voice-packet.ts`). No DOM, no Electron, no Node imports here.
- `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`. - `src/lib/native.ts`: `desktop` is the Electron preload API or null; `native` is what both platforms provide (identities, certificates), backed by `web.svelte.ts` in the browser build (`isWeb`, vite `--mode web`). Desktop-only features check `desktop`.
- `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text. - `src/lib/`: app state. `session.svelte.ts` has one `Session` per server plus the `sessions` manager; `session` is a Proxy to the active one. `audio/voice.svelte.ts` is the voice engine (WebCodecs Opus, capture and playback AudioWorklets). `html.ts` sanitizes incoming HTML and serializes outgoing rich text.
+3 -3
View File
@@ -78,11 +78,11 @@ mumh5 keeps the foundation and replaces the experience.
### Screen sharing ### Screen sharing
- Share a screen or a window with the mumh5 users in your channel. A dialog lets you choose what to show, with a live preview, which sound goes with it and the quality, before anything is sent - Share a screen, a window or a camera with the mumh5 users in your channel. A dialog lets you choose what to show, with a live preview, which sound goes with it and the quality, before anything is sent
- Sound: on Linux one program or everything except mumh5 itself (through PipeWire, so viewers do not hear the voice chat twice); on Windows the whole system; in a browser what the browser offers - Sound: on Linux one program or everything except mumh5 itself (through PipeWire, so viewers do not hear the voice chat twice); on Windows the whole system; in a browser what the browser offers
- People in the channel see an indicator next to your name, on your tile and in your profile, and click to watch. The tiles above the chat fill a stage you can drag taller or shorter. A stream opens large, with everyone as a strip of tiles below, like a meeting, and can move to a window of its own; in the stacked layout it is just the picture, as wide as the chat. Sounds announce streams and viewers. Viewers set the stream's volume - People in the channel see an indicator next to your name, on your tile and in your profile, and click to watch. The tiles above the chat fill a stage you can drag taller or shorter. A stream opens large, with everyone as a strip of tiles below, like a meeting, and can move to a window of its own; in the stacked layout it is just the picture, as wide as the chat. Sounds announce streams and viewers. Viewers set the stream's volume
- No server setup and no extra account: the setup messages travel through the Mumble server, the stream goes directly between the two clients (WebRTC), up to 8 viewers. Regular Mumble clients do not see streams - No server setup and no extra account: the setup messages travel through the Mumble server, the stream goes directly between the two clients (WebRTC), up to 8 viewers. Regular Mumble clients do not see streams
- Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Across the internet both sides need a STUN server. The browser version uses the one built into its proxy, and the proxy's relay when no direct connection is possible (mobile networks, Vanadium); in the desktop app you choose one in Settings, Voice (Google, Cloudflare, or any address such as your proxy's), and none is contacted unless you do - Direct connections mean sharer and viewer see each other's IP address; mumh5 says so before the first use. Across the internet both sides need a STUN server. The browser version uses the one built into its proxy, and the proxy's relay when no direct connection is possible (mobile networks, Vanadium); in the desktop app you choose a STUN server in Settings, Voice (Google, Cloudflare, or any address), and can enter the address of a mumh5 web version as a relay, which then takes over by itself when no direct connection comes up. Nothing is contacted unless you set it
- Tested between two desktop instances on one machine with a test picture. Sound capture, real screens, connections across the internet, Windows and the browser build are untested - Tested between two desktop instances on one machine with a test picture. Sound capture, real screens, connections across the internet, Windows and the browser build are untested
### Chat ### Chat
@@ -177,7 +177,7 @@ docker compose up -d --build
The container uses the host's network: the proxy listens on `127.0.0.1:8080` for your reverse proxy, and a Mumble server on the same machine is reachable as `localhost`. The container uses the host's network: the proxy listens on `127.0.0.1:8080` for your reverse proxy, and a Mumble server on the same machine is reachable as `localhost`.
For screen sharing, open these in the firewall (and forward them on a router in front of the server): port 3478 for UDP and TCP, and UDP 49160-49359. They carry STUN and the relay, which the browser version uses without any setting. Relayed streams pass through your server and use its bandwidth, a few Mbit/s per viewer; only people using your site get credentials for it, and there is no bandwidth cap yet. For screen sharing, open these in the firewall (and forward them on a router in front of the server): port 3478 for UDP and TCP, and UDP 49160-49359. They carry STUN and the relay, which the browser version uses without any setting. Relayed streams pass through your server and use its bandwidth, a few Mbit/s per viewer; anyone who knows the site's address can get credentials for it (the desktop app asks from outside the site), limited by connection quotas per address; there is no bandwidth cap yet.
| Variable | Default | Meaning | | Variable | Default | Meaning |
| --- | --- | --- | | --- | --- | --- |
+19 -11
View File
@@ -167,6 +167,20 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; s
if (route === 'config' && req.method === 'GET') { if (route === 'config' && req.method === 'GET') {
return { servers: config.servers, any: config.allowAny, stun: stunPort, turn: stunPort != null && config.turn }; return { servers: config.servers, any: config.allowAny, stun: stunPort, turn: stunPort != null && config.turn };
} }
// What a client needs for screen sharing: the STUN port and, with the relay on, credentials
// for it. Open to any origin, because the desktop app asks from outside this site; checking
// the origin would not keep a script out anyway. The quotas are what limit use.
if (route === 'relay' && req.method === 'GET') {
if (stunPort == null) throw new HttpError(404, 'No STUN or relay here');
if (!config.turn) return { port: stunPort };
const addr = addressOf(req);
const used = (identityUse.get(addr) ?? 0) + 1;
identityUse.set(addr, used);
if (used > 20) throw new HttpError(429, 'Too many requests, try again in a minute');
// The username is the time it runs out; the relay recomputes the password from it
const username = String(Math.floor(Date.now() / 1000) + CREDENTIAL_TTL);
return { port: stunPort, username, credential: turnCredential(turnSecret, username), ttl: CREDENTIAL_TTL };
}
if (req.method !== 'POST') throw new HttpError(404, 'Not found'); if (req.method !== 'POST') throw new HttpError(404, 'Not found');
if (!originOk(req)) throw new HttpError(403, 'Origin not allowed'); if (!originOk(req)) throw new HttpError(403, 'Origin not allowed');
const body = await readJson(req); const body = await readJson(req);
@@ -176,16 +190,6 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; s
identityUse.set(addr, used); identityUse.set(addr, used);
if (used > 20) throw new HttpError(429, 'Too many requests, try again in a minute'); if (used > 20) throw new HttpError(429, 'Too many requests, try again in a minute');
} }
if (route === 'turn') {
if (stunPort == null || !config.turn) throw new HttpError(404, 'No relay here');
const addr = addressOf(req);
const used = (identityUse.get(addr) ?? 0) + 1;
identityUse.set(addr, used);
if (used > 20) throw new HttpError(429, 'Too many requests, try again in a minute');
// The username is the time it runs out; the relay recomputes the password from it
const username = String(Math.floor(Date.now() / 1000) + CREDENTIAL_TTL);
return { username, credential: turnCredential(turnSecret, username), ttl: CREDENTIAL_TTL, port: stunPort };
}
switch (route) { switch (route) {
// Nothing is stored here: the browser keeps its identities and sends one along when it connects // Nothing is stored here: the browser keeps its identities and sends one along when it connects
case 'identity/create': case 'identity/create':
@@ -234,7 +238,11 @@ export async function startProxy(config: ProxyConfig): Promise<{ port: number; s
const work = pathname.startsWith('/api/') const work = pathname.startsWith('/api/')
? api(req, pathname.slice(5)).then(result => { ? api(req, pathname.slice(5)).then(result => {
const data = Buffer.from(JSON.stringify(result)); const data = Buffer.from(JSON.stringify(result));
res.writeHead(200, { 'Content-Type': 'application/json', 'Content-Length': data.length, 'Cache-Control': 'no-store', 'X-Content-Type-Options': 'nosniff' }); res.writeHead(200, {
'Content-Type': 'application/json', 'Content-Length': data.length, 'Cache-Control': 'no-store', 'X-Content-Type-Options': 'nosniff',
// The desktop app reads this one from another origin
...(pathname === '/api/relay' ? { 'Access-Control-Allow-Origin': '*' } : {})
});
res.end(data); res.end(data);
}) })
: serveStatic(req, res, pathname); : serveStatic(req, res, pathname);
+1 -1
View File
@@ -69,7 +69,7 @@ export function userMenu(u: User): MenuItem[] {
); );
if (u.userId == null) items.push({ label: 'Register on this server', action: () => session.register(u.session) }); if (u.userId == null) items.push({ label: 'Register on this server', action: () => session.register(u.session) });
} else { } else {
if (session.sharing[u.session]) items.push({ label: 'Watch screen', action: () => share.watch(sessions.active, u.session) }); if (session.sharing[u.session]) items.push({ label: `Watch ${session.sharing[u.session]}`, action: () => share.watch(sessions.active, u.session) });
items.push( items.push(
{ label: 'Send message', action: () => session.selectView(`dm:${u.session}`) }, { label: 'Send message', action: () => session.selectView(`dm:${u.session}`) },
{ label: 'Mute for me', checked: !!voice.settings.localMutes[userKey(u)], action: () => voice.setLocalMute(u, !voice.settings.localMutes[userKey(u)]) }, { label: 'Mute for me', checked: !!voice.settings.localMutes[userKey(u)], action: () => voice.setLocalMute(u, !voice.settings.localMutes[userKey(u)]) },
+1
View File
@@ -6,6 +6,7 @@ export const ICON_PATHS: Record<string, string> = {
'headphones-off': 'M3 14v-2a9 9 0 0 1 18 0v2M21 14v7h-4v-7zM3 14v7h4v-7zM3 3l18 18', 'headphones-off': 'M3 14v-2a9 9 0 0 1 18 0v2M21 14v7h-4v-7zM3 14v7h4v-7zM3 3l18 18',
settings: 'M4 21v-7M4 10V3M12 21v-9M12 8V3M20 21v-5M20 12V3M1 14h6M9 8h6M17 16h6', settings: 'M4 21v-7M4 10V3M12 21v-9M12 8V3M20 21v-5M20 12V3M1 14h6M9 8h6M17 16h6',
plus: 'M12 5v14M5 12h14', plus: 'M12 5v14M5 12h14',
camera: 'M2 6h14v12H2zM16 10l6-3v10l-6-3',
screen: 'M3 4h18v12H3zM8 20h8M12 16v4', screen: 'M3 4h18v12H3zM8 20h8M12 16v4',
popout: 'M14 4h6v6M20 4l-9 9M18 14v6H4V6h6', popout: 'M14 4h6v6M20 4l-9 9M18 14v6H4V6h6',
expand: 'M4 9V4h5M20 9V4h-5M4 15v5h5M20 15v5h-5', expand: 'M4 9V4h5M20 9V4h-5M4 15v5h5M20 15v5h-5',
+4 -4
View File
@@ -1,7 +1,7 @@
import { createCodec } from '../core/proto.ts'; import { createCodec } from '../core/proto.ts';
import { MumbleClient, describeDenial, type Channel, type User, type TextMessage } from '../core/client.ts'; import { MumbleClient, describeDenial, type Channel, type User, type TextMessage } from '../core/client.ts';
import { native, openTransport } from './native.ts'; import { native, openTransport } from './native.ts';
import { share } from './share.svelte.ts'; import { share, type ShareKind } from './share.svelte.ts';
import type { ServerCertInfo } from '../core/transport.ts'; import type { ServerCertInfo } from '../core/transport.ts';
import { store, type SavedServer } from './settings.svelte.ts'; import { store, type SavedServer } from './settings.svelte.ts';
import { renderIncoming, textToHtml, escapeHtml, formatBytes, type Embed } from './html.ts'; import { renderIncoming, textToHtml, escapeHtml, formatBytes, type Embed } from './html.ts';
@@ -62,7 +62,7 @@ export class Session {
certPrompt = $state<CertPrompt | null>(null); certPrompt = $state<CertPrompt | null>(null);
rtt = $state(0); rtt = $state(0);
// Who in our channel is sharing their screen, by session (mumh5 clients only) // Who in our channel is sharing their screen, by session (mumh5 clients only)
sharing = $state<Record<number, boolean>>({}); sharing = $state<Record<number, ShareKind>>({});
// Why the last connection ended, shown with a Reconnect button (null after a normal disconnect) // Why the last connection ended, shown with a Reconnect button (null after a normal disconnect)
disconnectInfo = $state<{ kind: 'kicked' | 'banned' | 'lost' | 'rejected'; by: string; reason: string } | null>(null); disconnectInfo = $state<{ kind: 'kicked' | 'banned' | 'lost' | 'rejected'; by: string; reason: string } | null>(null);
private removedBy: { kind: 'kicked' | 'banned'; by: string; reason: string } | null = null; private removedBy: { kind: 'kicked' | 'banned'; by: string; reason: string } | null = null;
@@ -687,8 +687,8 @@ export class Session {
// ─── Voice state and moderation ───────────────────────────────────────────── // ─── Voice state and moderation ─────────────────────────────────────────────
setSharing(session: number, on: boolean): void { setSharing(session: number, kind: ShareKind | false): void {
if (on) this.sharing[session] = true; if (kind) this.sharing[session] = kind;
else delete this.sharing[session]; else delete this.sharing[session];
} }
clearSharing(): void { this.sharing = {}; } clearSharing(): void { this.sharing = {}; }
+2
View File
@@ -26,6 +26,7 @@ export interface Settings {
arrangement: Arrangement; arrangement: Arrangement;
shareQuality: 'low' | 'medium' | 'high'; // bitrate and frame rate of our own stream shareQuality: 'low' | 'medium' | 'high'; // bitrate and frame rate of our own stream
shareVolume: number; // volume of streams we watch, 0 to 1 shareVolume: number; // volume of streams we watch, 0 to 1
relayServer: string; // a mumh5 web proxy whose STUN and relay to use for screen sharing
stunServer: string; // helps screen sharing connect through routers; empty contacts nobody stunServer: string; // helps screen sharing connect through routers; empty contacts nobody
shareNoticeSeen: boolean; // the note that screen sharing reveals IP addresses was confirmed shareNoticeSeen: boolean; // the note that screen sharing reveals IP addresses was confirmed
hints: boolean; // tooltips (title attributes) when resting the pointer on buttons hints: boolean; // tooltips (title attributes) when resting the pointer on buttons
@@ -58,6 +59,7 @@ const defaults: Settings = {
arrangement: 'side', arrangement: 'side',
shareQuality: 'medium', shareQuality: 'medium',
shareVolume: 1, shareVolume: 1,
relayServer: '',
stunServer: '', stunServer: '',
shareNoticeSeen: false, shareNoticeSeen: false,
hints: false hints: false
+22 -14
View File
@@ -6,7 +6,7 @@ import { SHARE, SHARE_DATA_ID, encodeShare, ShareAssembler, type ShareType } fro
import type { User } from '../core/client.ts'; import type { User } from '../core/client.ts';
import type { Session } from './session.svelte.ts'; import type { Session } from './session.svelte.ts';
import { desktop, isWeb } from './native.ts'; import { desktop, isWeb } from './native.ts';
import { proxyIce } from './web.svelte.ts'; import { proxyIce, relayIce } from './web.svelte.ts';
import { store } from './settings.svelte.ts'; import { store } from './settings.svelte.ts';
import { ui } from './ui.svelte.ts'; import { ui } from './ui.svelte.ts';
import { sounds } from './audio/sounds.svelte.ts'; import { sounds } from './audio/sounds.svelte.ts';
@@ -23,6 +23,9 @@ export const QUALITY = {
const stereo = (sdp: string) => sdp.replace(/a=fmtp:(\d+) ([^\r\n]*useinbandfec=1[^\r\n]*)/g, const stereo = (sdp: string) => sdp.replace(/a=fmtp:(\d+) ([^\r\n]*useinbandfec=1[^\r\n]*)/g,
(line, pt, rest) => rest.includes('stereo=1') ? line : `a=fmtp:${pt} ${rest};stereo=1;sprop-stereo=1;maxaveragebitrate=192000`); (line, pt, rest) => rest.includes('stereo=1') ? line : `a=fmtp:${pt} ${rest};stereo=1;sprop-stereo=1;maxaveragebitrate=192000`);
// What a stream shows. Sent along with the announcement, so others see the right icon.
export type ShareKind = 'screen' | 'camera';
export interface Watching { export interface Watching {
host: Session; host: Session;
session: number; session: number;
@@ -34,6 +37,7 @@ class ScreenShare {
// The server we are sharing on, and what we send // The server we are sharing on, and what we send
host = $state.raw<Session | null>(null); host = $state.raw<Session | null>(null);
stream = $state.raw<MediaStream | null>(null); stream = $state.raw<MediaStream | null>(null);
kind = $state<ShareKind>('screen');
viewers = $state(0); viewers = $state(0);
watching = $state.raw<Watching | null>(null); watching = $state.raw<Watching | null>(null);
error = $state(''); error = $state('');
@@ -58,9 +62,11 @@ class ScreenShare {
private async iceServers(): Promise<RTCIceServer[]> { private async iceServers(): Promise<RTCIceServer[]> {
const stun = store.settings.stunServer.trim(); const stun = store.settings.stunServer.trim();
const chosen: RTCIceServer[] = stun ? [{ urls: /^stuns?:/.test(stun) ? stun : `stun:${stun}` }] : []; const chosen: RTCIceServer[] = stun ? [{ urls: /^stuns?:/.test(stun) ? stun : `stun:${stun}` }] : [];
// The browser build also has the proxy it is served from: its STUN, and its relay for // A mumh5 proxy adds its STUN and its relay, which takes over by itself when no direct
// networks and browsers that allow no direct connection // connection comes up (mobile and company networks, browsers that forbid direct UDP). The
return isWeb ? [...chosen, ...await proxyIce()] : chosen; // browser build has the proxy it is served from; the desktop app the one set in Settings.
const relay = store.settings.relayServer.trim();
return [...chosen, ...(relay ? await relayIce(relay) : isWeb ? await proxyIce() : [])];
} }
// For testing the relay: localStorage mumh5.iceDebug = "relay" uses only relayed routes, // For testing the relay: localStorage mumh5.iceDebug = "relay" uses only relayed routes,
@@ -112,7 +118,7 @@ class ScreenShare {
} }
// The dialog hands over what it captured // The dialog hands over what it captured
go(s: Session, stream: MediaStream, usesVirtualMic: boolean): void { go(s: Session, stream: MediaStream, usesVirtualMic: boolean, kind: ShareKind = 'screen'): void {
this.dialog = null; this.dialog = null;
// Disconnected while the dialog was open // Disconnected while the dialog was open
if (!s.client || this.stream) { if (!s.client || this.stream) {
@@ -121,15 +127,17 @@ class ScreenShare {
return; return;
} }
for (const t of stream.getVideoTracks()) { for (const t of stream.getVideoTracks()) {
t.contentHint = 'detail'; // Text on a screen must stay sharp; a face may blur a little but should move smoothly
t.contentHint = kind === 'camera' ? 'motion' : 'detail';
// The system's "stop sharing" control // The system's "stop sharing" control
t.addEventListener('ended', () => this.stop()); t.addEventListener('ended', () => this.stop());
} }
this.usesVirtualMic = usesVirtualMic; this.usesVirtualMic = usesVirtualMic;
this.kind = kind;
this.host = s; this.host = s;
this.stream = stream; this.stream = stream;
this.send(s, this.others(s), SHARE.on); this.send(s, this.others(s), SHARE.on, kind);
s.setSharing(s.client.session!, true); s.setSharing(s.client.session!, kind);
sounds.play('shareStarted'); sounds.play('shareStarted');
} }
@@ -228,7 +236,7 @@ class ScreenShare {
if (track.kind !== 'video') continue; if (track.kind !== 'video') continue;
const params = sender.getParameters(); const params = sender.getParameters();
// Keep the picture sharp and let the frame rate drop when the connection is tight // Keep the picture sharp and let the frame rate drop when the connection is tight
params.degradationPreference = 'maintain-resolution'; params.degradationPreference = this.kind === 'camera' ? 'maintain-framerate' : 'maintain-resolution';
for (const enc of params.encodings ?? []) enc.maxBitrate = QUALITY[store.settings.shareQuality].bitrate; for (const enc of params.encodings ?? []) enc.maxBitrate = QUALITY[store.settings.shareQuality].bitrate;
await sender.setParameters(params).catch(() => {}); await sender.setParameters(params).catch(() => {});
} }
@@ -283,14 +291,14 @@ class ScreenShare {
switch (msg.type) { switch (msg.type) {
case SHARE.on: case SHARE.on:
if (!s.sharing[sender] && Date.now() - (this.enteredAt.get(s) ?? 0) > 3000) sounds.play('shareStarted'); if (!s.sharing[sender] && Date.now() - (this.enteredAt.get(s) ?? 0) > 3000) sounds.play('shareStarted');
s.setSharing(sender, true); s.setSharing(sender, msg.text === 'camera' ? 'camera' : 'screen');
break; break;
case SHARE.off: case SHARE.off:
if (s.sharing[sender]) sounds.play('shareStopped'); if (s.sharing[sender]) sounds.play('shareStopped');
s.setSharing(sender, false); s.setSharing(sender, false);
if (this.watching?.host === s && this.watching.session === sender) this.endWatch(); if (this.watching?.host === s && this.watching.session === sender) this.endWatch();
break; break;
case SHARE.who: if (this.host === s) this.send(s, [sender], SHARE.on); break; case SHARE.who: if (this.host === s) this.send(s, [sender], SHARE.on, this.kind); break;
case SHARE.watch: if (this.host === s) this.offerTo(s, sender).catch(() => this.dropPeer(sender)); break; case SHARE.watch: if (this.host === s) this.offerTo(s, sender).catch(() => this.dropPeer(sender)); break;
case SHARE.leave: if (this.host === s) this.dropPeer(sender); break; case SHARE.leave: if (this.host === s) this.dropPeer(sender); break;
case SHARE.offer: if (msg.text) this.answerTo(s, sender, msg.text); break; case SHARE.offer: if (msg.text) this.answerTo(s, sender, msg.text); break;
@@ -309,8 +317,8 @@ class ScreenShare {
if (this.watching?.host === s) this.unwatch(); if (this.watching?.host === s) this.unwatch();
if (this.host === s) { if (this.host === s) {
this.dropPeers(); this.dropPeers();
s.setSharing(self, true); s.setSharing(self, this.kind);
this.send(s, this.others(s), SHARE.on); this.send(s, this.others(s), SHARE.on, this.kind);
} }
this.send(s, this.others(s), SHARE.who); this.send(s, this.others(s), SHARE.who);
} }
@@ -319,7 +327,7 @@ class ScreenShare {
userMoved(s: Session, user: User, gone = false): void { userMoved(s: Session, user: User, gone = false): void {
if (!s.client?.self || user.session === s.client.session) return; if (!s.client?.self || user.session === s.client.session) return;
if (!gone && this.inMyChannel(s, user.session)) { if (!gone && this.inMyChannel(s, user.session)) {
if (this.host === s) this.send(s, [user.session], SHARE.on); if (this.host === s) this.send(s, [user.session], SHARE.on, this.kind);
return; return;
} }
// A sharer who left or disconnected: their stream is over for us // A sharer who left or disconnected: their stream is over for us
+18 -18
View File
@@ -137,8 +137,6 @@ class ProxyInfo {
any = $state(false); any = $state(false);
// UDP port of the proxy's STUN responder, if it runs one // UDP port of the proxy's STUN responder, if it runs one
stun = $state<number | null>(null); stun = $state<number | null>(null);
// Whether it also relays streams for browsers that cannot connect directly
turn = $state(false);
loaded = $state(false); loaded = $state(false);
error = $state(''); error = $state('');
@@ -148,7 +146,6 @@ class ProxyInfo {
this.servers = c.servers; this.servers = c.servers;
this.any = c.any; this.any = c.any;
this.stun = c.stun ?? null; this.stun = c.stun ?? null;
this.turn = !!c.turn;
this.error = ''; this.error = '';
} catch (e) { } catch (e) {
this.error = (e as Error).message; this.error = (e as Error).message;
@@ -158,24 +155,27 @@ class ProxyInfo {
} }
export const proxyInfo = new ProxyInfo(); export const proxyInfo = new ProxyInfo();
// STUN and relay on the proxy itself: a host the user already uses, so nothing new is contacted. // STUN and the relay of a mumh5 proxy, for screen sharing. The browser build uses the proxy it
// The relay needs credentials, which the proxy hands out for a few hours at a time. // is served from, a host the user already uses; the desktop app one the user entered. The relay
let relay: { servers: RTCIceServer[]; until: number } | null = null; // needs credentials, which the proxy hands out for a few hours at a time.
export async function proxyIce(): Promise<RTCIceServer[]> { const relays = new Map<string, { servers: RTCIceServer[]; until: number }>();
if (!proxyInfo.loaded) await proxyInfo.load(); export async function relayIce(address: string): Promise<RTCIceServer[]> {
if (!proxyInfo.stun || !base) return []; let url: URL;
let host: string; try { url = new URL(/^https?:\/\//i.test(address) ? address : `https://${address}`); } catch { return []; }
try { host = new URL(base).hostname; } catch { return []; } const root = url.href.replace(/\/*$/, '/');
const stun: RTCIceServer = { urls: `stun:${host}:${proxyInfo.stun}` }; const known = relays.get(root);
if (!proxyInfo.turn) return [stun]; if (known && known.until > Date.now()) return known.servers;
if (relay && relay.until > Date.now()) return relay.servers;
try { try {
const c = await call<{ username: string; credential: string; ttl: number; port: number }>('turn', {}); const res = await fetch(`${root}api/relay`);
if (!res.ok) return [];
const c = await res.json() as { port: number; username?: string; credential?: string; ttl?: number };
const servers: RTCIceServer[] = [{ urls: `stun:${url.hostname}:${c.port}` }];
// UDP where it is allowed, TCP for networks and browsers that forbid it // UDP where it is allowed, TCP for networks and browsers that forbid it
const servers = [stun, { urls: [`turn:${host}:${c.port}?transport=udp`, `turn:${host}:${c.port}?transport=tcp`], username: c.username, credential: c.credential }]; if (c.username) servers.push({ urls: [`turn:${url.hostname}:${c.port}?transport=udp`, `turn:${url.hostname}:${c.port}?transport=tcp`], username: c.username, credential: c.credential });
relay = { servers, until: Date.now() + Math.max(60, c.ttl - 3600) * 1000 }; relays.set(root, { servers, until: Date.now() + Math.max(60, (c.ttl ?? 3600) - 3000) * 1000 });
return servers; return servers;
} catch { } catch {
return [stun]; return [];
} }
} }
export const proxyIce = (): Promise<RTCIceServer[]> => base ? relayIce(base) : Promise.resolve([]);
+4 -3
View File
@@ -169,14 +169,15 @@
<span class="label" class:talking={voice.talking[u.session] || (isSelf && voice.transmitting)}>{store.displayName(u)}</span> <span class="label" class:talking={voice.talking[u.session] || (isSelf && voice.transmitting)}>{store.displayName(u)}</span>
{#if u.userId != null}<span class="reg" title="Registered">R</span>{/if} {#if u.userId != null}<span class="reg" title="Registered">R</span>{/if}
{#if session.sharing[u.session]} {#if session.sharing[u.session]}
{@const kind = session.sharing[u.session]}
{#if isSelf} {#if isSelf}
<span class="state sharing" title="You are sharing your screen"><Icon name="screen" size={14} /></span> <span class="state sharing" title="You are sharing your {kind}"><Icon name={kind} size={14} /></span>
{:else} {:else}
<!-- A button cannot sit inside the row's button --> <!-- A button cannot sit inside the row's button -->
<span class="state sharing" role="button" tabindex="0" title="Sharing their screen. Click to watch" aria-label="Watch the screen of {store.displayName(u)}" <span class="state sharing" role="button" tabindex="0" title="Sharing their {kind}. Click to watch" aria-label="Watch the {kind} of {store.displayName(u)}"
onclick={e => { e.stopPropagation(); clearTimeout(timer); share.watch(sessions.active, u.session); }} onclick={e => { e.stopPropagation(); clearTimeout(timer); share.watch(sessions.active, u.session); }}
ondblclick={e => e.stopPropagation()} ondblclick={e => e.stopPropagation()}
onkeydown={e => { if (e.key === 'Enter') { e.stopPropagation(); share.watch(sessions.active, u.session); } }}><Icon name="screen" size={14} /></span> onkeydown={e => { if (e.key === 'Enter') { e.stopPropagation(); share.watch(sessions.active, u.session); } }}><Icon name={kind} size={14} /></span>
{/if} {/if}
{/if} {/if}
{#if u.comment || u.commentHash}<span class="state" title="Has a description"><Icon name="message" size={12} /></span>{/if} {#if u.comment || u.commentHash}<span class="state" title="Has a description"><Icon name="message" size={12} /></span>{/if}
+2 -2
View File
@@ -128,8 +128,8 @@
{#if session.sharing[user.session]} {#if session.sharing[user.session]}
<div class="sharing"> <div class="sharing">
<Icon name="screen" size={16} /> <Icon name={session.sharing[user.session]} size={16} />
<span>{isSelf ? 'You are sharing your screen' : 'Sharing their screen'}</span> <span>{isSelf ? `You are sharing your ${session.sharing[user.session]}` : `Sharing their ${session.sharing[user.session]}`}</span>
{#if isSelf}<button class="btn" onclick={() => share.stop()}>Stop sharing</button> {#if isSelf}<button class="btn" onclick={() => share.stop()}>Stop sharing</button>
{:else if share.watching?.session === user.session}<button class="btn" onclick={() => share.unwatch()}>Stop watching</button> {:else if share.watching?.session === user.session}<button class="btn" onclick={() => share.unwatch()}>Stop watching</button>
{:else}<button class="btn primary" onclick={() => share.watch(sessions.active, user.session)}>Watch</button>{/if} {:else}<button class="btn primary" onclick={() => share.watch(sessions.active, user.session)}>Watch</button>{/if}
+5
View File
@@ -84,6 +84,10 @@
{#if stunChoice === 'custom'} {#if stunChoice === 'custom'}
<input class="stun-custom" bind:value={s.stunServer} onchange={() => store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" aria-label="STUN server address" /> <input class="stun-custom" bind:value={s.stunServer} onchange={() => store.saveSettings()} placeholder="stun.example.com:3478" autocomplete="off" aria-label="STUN server address" />
{/if} {/if}
<label for="s-relay">Relay <span class="opt">(optional)</span></label>
<input id="s-relay" bind:value={s.relayServer} onchange={() => { s.relayServer = s.relayServer.trim(); store.saveSettings(); }}
placeholder={isWeb ? 'this site (automatic)' : 'https://voice.example.com'} autocomplete="off" />
<p class="help">The address of a mumh5 web version. When a direct connection is not possible (mobile data, strict company networks), the stream goes through that server instead, by itself. {isWeb ? 'Empty uses the site you are on.' : 'Its operator sees your IP address and carries the stream, but cannot watch it.'}</p>
<p class="help">Screen sharing connects you directly to the people watching. Across the internet both sides need a STUN server to find a route; it learns your IP address but never sees the stream. {isWeb ? 'By default the site you are on provides it.' : 'With none, no outside server is contacted and streams only work on the same network.'} Networks that block direct connections, like many mobile ones, need more than STUN.</p> <p class="help">Screen sharing connects you directly to the people watching. Across the internet both sides need a STUN server to find a route; it learns your IP address but never sees the stream. {isWeb ? 'By default the site you are on provides it.' : 'With none, no outside server is contacted and streams only work on the same network.'} Networks that block direct connections, like many mobile ones, need more than STUN.</p>
{:else if tab === 'sounds'} {:else if tab === 'sounds'}
<SoundSettings /> <SoundSettings />
@@ -146,6 +150,7 @@
.tabs button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); } .tabs button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); }
h3 { margin: 22px 0 0; font-size: 14px; } h3 { margin: 22px 0 0; font-size: 14px; }
.stun-custom { margin-top: 6px; } .stun-custom { margin-top: 6px; }
.opt { text-transform: none; font-weight: 400; letter-spacing: 0; color: var(--text-faint); }
.help { margin: 10px 0 0; font-size: 13px; color: var(--text-dim); } .help { margin: 10px 0 0; font-size: 13px; color: var(--text-dim); }
.row { display: flex; align-items: flex-end; gap: 8px; } .row { display: flex; align-items: flex-end; gap: 8px; }
.row > div { flex: 1; } .row > div { flex: 1; }
+56 -6
View File
@@ -1,7 +1,7 @@
<script lang="ts"> <script lang="ts">
import { onDestroy } from 'svelte'; import { onDestroy } from 'svelte';
import Modal from './Modal.svelte'; import Modal from './Modal.svelte';
import { share, QUALITY } from '../lib/share.svelte.ts'; import { share, QUALITY, type ShareKind } from '../lib/share.svelte.ts';
import { desktop, type ScreenSource } from '../lib/native.ts'; import { desktop, type ScreenSource } from '../lib/native.ts';
import { store } from '../lib/settings.svelte.ts'; import { store } from '../lib/settings.svelte.ts';
@@ -12,6 +12,12 @@
const target = share.dialog!; const target = share.dialog!;
let info = $state<{ portal: boolean; audio: 'pipewire' | 'loopback' | 'browser' | 'none'; micLabel: string }>({ portal: true, audio: desktop ? 'none' : 'browser', micLabel: '' }); let info = $state<{ portal: boolean; audio: 'pipewire' | 'loopback' | 'browser' | 'none'; micLabel: string }>({ portal: true, audio: desktop ? 'none' : 'browser', micLabel: '' });
// What to show: the screen (or a window), or a camera
// Phones have no screen capture in the browser; there the camera is the only choice
const canScreen = !!navigator.mediaDevices?.getDisplayMedia;
let kind = $state<ShareKind>(canScreen ? 'screen' : 'camera');
let cameras = $state<MediaDeviceInfo[]>([]);
let camera = $state('');
let sources = $state<ScreenSource[]>([]); let sources = $state<ScreenSource[]>([]);
let selected = $state(''); let selected = $state('');
let preview = $state.raw<MediaStream | null>(null); let preview = $state.raw<MediaStream | null>(null);
@@ -67,6 +73,32 @@
} }
} }
async function showCamera(deviceId = camera) {
error = '';
dropPreview();
try {
preview = await navigator.mediaDevices.getUserMedia({
video: { deviceId: deviceId ? { exact: deviceId } : undefined, width: { ideal: 1280 }, height: { ideal: 720 }, frameRate: fps() }
});
// Names are only given out once a camera was allowed
cameras = (await navigator.mediaDevices.enumerateDevices()).filter(d => d.kind === 'videoinput');
camera = preview.getVideoTracks()[0]?.getSettings().deviceId ?? deviceId;
} catch (e) {
error = (e as Error).name === 'NotFoundError' ? 'No camera was found.' : `Could not open the camera: ${(e as Error).message}`;
}
}
if (!canScreen) showCamera();
function setKind(next: ShareKind) {
if (kind === next) return;
kind = next;
selected = '';
dropPreview();
error = '';
if (next === 'camera') showCamera();
}
// The system's or the browser's picker // The system's or the browser's picker
async function chooseElsewhere() { async function chooseElsewhere() {
error = ''; error = '';
@@ -106,7 +138,9 @@
error = ''; error = '';
let usesVirtualMic = false; let usesVirtualMic = false;
try { try {
if (audio && info.audio === 'pipewire') { // A camera comes without extra sound: your voice already goes through the voice chat
if (kind === 'camera') { /* nothing to add */ }
else if (audio && info.audio === 'pipewire') {
usesVirtualMic = true; usesVirtualMic = true;
await desktop!.shareAudioStart(audio === 'all' ? { kind: 'all' } : { kind: 'app', name: audio.slice(4) }); await desktop!.shareAudioStart(audio === 'all' ? { kind: 'all' } : { kind: 'app', name: audio.slice(4) });
preview.addTrack(await virtualMic()); preview.addTrack(await virtualMic());
@@ -125,7 +159,7 @@
return; return;
} }
handedOver = true; handedOver = true;
share.go(target, preview, usesVirtualMic); share.go(target, preview, usesVirtualMic, kind);
onclose(); onclose();
} }
@@ -135,11 +169,24 @@
<Modal title="Share your screen" {onclose} width={760}> <Modal title="Share your screen" {onclose} width={760}>
<p class="help">mumh5 users in your channel can watch. Nothing is sent until you press Start sharing.</p> <p class="help">mumh5 users in your channel can watch. Nothing is sent until you press Start sharing.</p>
{#if !desktop && info.audio === 'browser'} <div class="kinds" role="tablist" aria-label="What to share">
<button role="tab" aria-selected={kind === 'screen'} class:active={kind === 'screen'} disabled={!canScreen}
title={canScreen ? undefined : 'This browser cannot capture the screen'} onclick={() => setKind('screen')}>Screen or window</button>
<button role="tab" aria-selected={kind === 'camera'} class:active={kind === 'camera'} onclick={() => setKind('camera')}>Camera</button>
</div>
{#if kind === 'camera'}
<label for="sh-cam">Camera</label>
<select id="sh-cam" value={camera} onchange={e => showCamera(e.currentTarget.value)} disabled={!cameras.length}>
{#each cameras as c, i (c.deviceId)}<option value={c.deviceId}>{c.label || `Camera ${i + 1}`}</option>{:else}<option value="">{error ? 'None' : 'Asking for the camera...'}</option>{/each}
</select>
{:else if !desktop && info.audio === 'browser'}
<label class="check"><input type="checkbox" bind:checked={browserAudio} disabled={!!preview} /> Include sound (the browser offers what it can share, usually a tab's sound)</label> <label class="check"><input type="checkbox" bind:checked={browserAudio} disabled={!!preview} /> Include sound (the browser offers what it can share, usually a tab's sound)</label>
{/if} {/if}
{#if desktop && !info.portal} {#if kind === 'camera'}
<!-- chosen above -->
{:else if desktop && !info.portal}
<div class="row head"><label for="sh-src">Screen or window</label><button class="link" onclick={loadSources}>Refresh</button></div> <div class="row head"><label for="sh-src">Screen or window</label><button class="link" onclick={loadSources}>Refresh</button></div>
<ul class="sources" id="sh-src"> <ul class="sources" id="sh-src">
{#each sources as s (s.id)} {#each sources as s (s.id)}
@@ -165,7 +212,7 @@
</div> </div>
<div class="row two"> <div class="row two">
{#if desktop} {#if desktop && kind === 'screen'}
<div> <div>
<label for="sh-audio">Sound</label> <label for="sh-audio">Sound</label>
{#if info.audio === 'pipewire'} {#if info.audio === 'pipewire'}
@@ -202,6 +249,9 @@
<style> <style>
.help { margin: 8px 0 0; font-size: 13px; color: var(--text-dim); } .help { margin: 8px 0 0; font-size: 13px; color: var(--text-dim); }
.kinds { display: flex; gap: 2px; margin-top: 12px; border-bottom: 1px solid var(--line); }
.kinds button { padding: 6px 14px; color: var(--text-dim); font-weight: 600; }
.kinds button.active { color: var(--text); box-shadow: inset 0 -2px 0 var(--accent); }
.row { display: flex; align-items: center; gap: 8px; margin-top: 12px; } .row { display: flex; align-items: center; gap: 8px; margin-top: 12px; }
.row.head { justify-content: space-between; margin-top: 0; } .row.head { justify-content: space-between; margin-top: 0; }
.row.two { align-items: flex-end; gap: 12px; } .row.two { align-items: flex-end; gap: 12px; }
+3 -3
View File
@@ -168,9 +168,9 @@
onclick={() => session.setSelfDeaf(!deafened)} oncontextmenu={e => openQuick(e, 'output')}> onclick={() => session.setSelfDeaf(!deafened)} oncontextmenu={e => openQuick(e, 'output')}>
<Icon name={deafened ? 'headphones-off' : 'headphones'} size={18} /> <Icon name={deafened ? 'headphones-off' : 'headphones'} size={18} />
</button> </button>
<button class="icon-btn" class:live={sharingHere} title={sharingHere ? 'Stop sharing your screen' : 'Share your screen with your channel'} <button class="icon-btn" class:live={sharingHere} title={sharingHere ? `Stop sharing your ${share.kind}` : 'Share your screen or camera with your channel'}
aria-label={sharingHere ? 'Stop sharing your screen' : 'Share your screen'} disabled={!sharingHere && !canShare} onclick={toggleShare}> aria-label={sharingHere ? `Stop sharing your ${share.kind}` : 'Share your screen or camera'} disabled={!sharingHere && !canShare} onclick={toggleShare}>
<Icon name="screen" size={18} /> <Icon name={sharingHere ? share.kind : 'screen'} size={18} />
</button> </button>
<span class="sep"></span> <span class="sep"></span>
<button class="icon-btn" title="Change your description" aria-label="Change your description" disabled={!self} onclick={editDescription}> <button class="icon-btn" title="Change your description" aria-label="Change your description" disabled={!self} onclick={editDescription}>
+12 -10
View File
@@ -43,6 +43,8 @@
let focus = $state<number | null>(null); let focus = $state<number | null>(null);
const focused = $derived(focus != null ? streamOf(focus) : null); const focused = $derived(focus != null ? streamOf(focus) : null);
const focusUser = $derived(focus != null ? session.user(focus) : undefined); const focusUser = $derived(focus != null ? session.user(focus) : undefined);
// Screen or camera, for the wording and the icons
const focusKind = $derived((focus != null && session.sharing[focus]) || 'screen');
// A stream we just started watching goes to the stage; it leaves when the stream ends // A stream we just started watching goes to the stage; it leaves when the stream ends
let lastWatch: number | null = null; let lastWatch: number | null = null;
$effect(() => { $effect(() => {
@@ -206,7 +208,7 @@
// The window follows the picture on the stage and closes with it // The window follows the picture on the stage and closes with it
$effect(() => { $effect(() => {
const stream = focused; const stream = focused;
const title = focus === self?.session ? 'Your screen' : focusUser ? store.displayName(focusUser) : 'Screen share'; const title = focus === self?.session ? `Your ${focusKind}` : focusUser ? store.displayName(focusUser) : 'Screen share';
if (!popped) return; if (!popped) return;
if (!stream) return popIn(); if (!stream) return popIn();
if (popVideo && popVideo.srcObject !== stream) popVideo.srcObject = stream; if (popVideo && popVideo.srcObject !== stream) popVideo.srcObject = stream;
@@ -221,8 +223,8 @@
{#if self && focus != null && focused} {#if self && focus != null && focused}
<div class="spot" role="region" aria-label="Screen share" bind:this={frame}> <div class="spot" role="region" aria-label="Screen share" bind:this={frame}>
<div class="spot-head"> <div class="spot-head">
<Icon name="screen" size={16} /> <Icon name={focusKind} size={16} />
<span class="spot-who">{focus === self.session ? 'Your screen' : focusUser ? store.displayName(focusUser) : 'Screen share'}</span> <span class="spot-who">{focus === self.session ? `Your ${focusKind}` : focusUser ? store.displayName(focusUser) : 'Screen share'}</span>
{#if focus !== self.session && hasSound} {#if focus !== self.session && hasSound}
<Icon name={store.settings.shareVolume === 0 ? 'headphones-off' : 'headphones'} size={16} /> <Icon name={store.settings.shareVolume === 0 ? 'headphones-off' : 'headphones'} size={16} />
<input class="vol" type="range" min="0" max="1" step="0.01" bind:value={store.settings.shareVolume} onchange={() => store.saveSettings()} <input class="vol" type="range" min="0" max="1" step="0.01" bind:value={store.settings.shareVolume} onchange={() => store.saveSettings()}
@@ -236,7 +238,7 @@
{/if} {/if}
{#if focus !== self.session}<button class="btn small" onclick={() => share.unwatch()}>Stop watching</button> {#if focus !== self.session}<button class="btn small" onclick={() => share.unwatch()}>Stop watching</button>
{:else if videoOnly} {:else if videoOnly}
<span class="status" role="status" title="People watching your screen">{share.viewers} watching</span> <span class="status" role="status" title="People watching your {share.kind}">{share.viewers} watching</span>
<button class="btn small" onclick={() => share.stop()}>Stop sharing</button> <button class="btn small" onclick={() => share.stop()}>Stop sharing</button>
{/if} {/if}
{#if !videoOnly}<button class="icon-btn" title="Back to tiles" aria-label="Back to tiles" onclick={() => (focus = null)}><Icon name="x" size={16} /></button>{/if} {#if !videoOnly}<button class="icon-btn" title="Back to tiles" aria-label="Back to tiles" onclick={() => (focus = null)}><Icon name="x" size={16} /></button>{/if}
@@ -250,7 +252,7 @@
<video bind:this={spotVideo} use:media={focused} autoplay playsinline muted ondblclick={fullscreen}></video> <video bind:this={spotVideo} use:media={focused} autoplay playsinline muted ondblclick={fullscreen}></video>
{#if focus !== self.session && watching && watching.state !== 'live'} {#if focus !== self.session && watching && watching.state !== 'live'}
<p class="pending" class:bad={watching.state === 'failed'}>{watching.state === 'failed' <p class="pending" class:bad={watching.state === 'failed'}>{watching.state === 'failed'
? 'Could not connect to the stream. Across the internet both of you need a STUN server; some mobile and company networks block direct connections altogether.' ? 'Could not connect to the stream. Across the internet both of you need a STUN server; networks that block direct connections need a relay. Both are in Settings, Voice.'
: 'Connecting to the stream...'}</p> : 'Connecting to the stream...'}</p>
{/if} {/if}
{#if showStats}<ShareStats own={focus === self.session} onclose={() => (showStats = false)} />{/if} {#if showStats}<ShareStats own={focus === self.session} onclose={() => (showStats = false)} />{/if}
@@ -294,7 +296,7 @@
{/if} {/if}
</button> </button>
{#if mine} {#if mine}
<span class="status" role="status" title="People watching your screen"><Icon name="screen" size={14} />{share.viewers} watching</span> <span class="status" role="status" title="People watching your {share.kind}"><Icon name={share.kind} size={14} />{share.viewers} watching</span>
<button class="btn small" onclick={() => share.stop()}>Stop sharing</button> <button class="btn small" onclick={() => share.stop()}>Stop sharing</button>
{/if} {/if}
<button class="icon-btn" class:on={self.selfMute} aria-label={self.selfMute ? 'Unmute' : 'Mute'} title={self.selfMute ? 'Unmute' : 'Mute'} <button class="icon-btn" class:on={self.selfMute} aria-label={self.selfMute ? 'Unmute' : 'Mute'} title={self.selfMute ? 'Unmute' : 'Mute'}
@@ -317,19 +319,19 @@
<div class="row" role="presentation"> <div class="row" role="presentation">
{#each row as u (u.session)} {#each row as u (u.session)}
{@const stream = streamOf(u.session)} {@const stream = streamOf(u.session)}
{@const sharing = !!session.sharing[u.session]} {@const sharing = session.sharing[u.session]}
{@const isSelf = u.session === self.session} {@const isSelf = u.session === self.session}
<button class="tile" class:talking={talking(u.session)} class:self={isSelf} class:focused={focus === u.session} <button class="tile" class:talking={talking(u.session)} class:self={isSelf} class:focused={focus === u.session}
onclick={() => tileClick(u.session)} onclick={() => tileClick(u.session)}
oncontextmenu={e => menu.show(e, userMenu(u))} oncontextmenu={e => menu.show(e, userMenu(u))}
title={stream ? (focus === u.session ? 'Hide the stream' : 'Show the stream') : sharing && !isSelf ? 'Sharing their screen. Click to watch' : store.displayName(u)} title={stream ? (focus === u.session ? 'Hide the stream' : 'Show the stream') : sharing && !isSelf ? `Sharing their ${sharing}. Click to watch` : store.displayName(u)}
aria-label={stream ? `${focus === u.session ? 'Hide' : 'Show'} the screen of ${store.displayName(u)}` : sharing && !isSelf ? `Watch the screen of ${store.displayName(u)}` : store.displayName(u)}> aria-label={stream ? `${focus === u.session ? 'Hide' : 'Show'} the ${sharing || 'stream'} of ${store.displayName(u)}` : sharing && !isSelf ? `Watch the ${sharing} of ${store.displayName(u)}` : store.displayName(u)}>
<Avatar name={u.name} size={avatar} speaking={talking(u.session)} /> <Avatar name={u.name} size={avatar} speaking={talking(u.session)} />
<span class="who">{store.displayName(u)}</span> <span class="who">{store.displayName(u)}</span>
{#if watching?.session === u.session && watching.state !== 'live'} {#if watching?.session === u.session && watching.state !== 'live'}
<span class="note">{watching.state === 'connecting' ? 'Connecting...' : 'Could not connect'}</span> <span class="note">{watching.state === 'connecting' ? 'Connecting...' : 'Could not connect'}</span>
{/if} {/if}
{#if sharing}<span class="badge live"><Icon name="screen" size={13} /></span> {#if sharing}<span class="badge live"><Icon name={sharing} size={13} /></span>
{:else if u.deaf || u.selfDeaf} {:else if u.deaf || u.selfDeaf}
<span class="badge" class:admin={u.deaf}><Icon name="headphones-off" size={13} /></span> <span class="badge" class:admin={u.deaf}><Icon name="headphones-off" size={13} /></span>
{:else if u.mute || u.selfMute || u.suppress} {:else if u.mute || u.selfMute || u.suppress}
+36
View File
@@ -4,6 +4,7 @@ import { _electron as electron, type Page } from 'playwright-core';
import electronPath from 'electron'; import electronPath from 'electron';
import assert from 'node:assert/strict'; import assert from 'node:assert/strict';
import { mkdtempSync } from 'node:fs'; import { mkdtempSync } from 'node:fs';
import { spawn } from 'node:child_process';
import { tmpdir } from 'node:os'; import { tmpdir } from 'node:os';
import path from 'node:path'; import path from 'node:path';
@@ -45,6 +46,7 @@ async function join(page: Page, name: string) {
await page.getByText(/Connected/).first().waitFor(); await page.getByText(/Connected/).first().waitFor();
} }
const cleanup: (() => void)[] = [];
const alice = await launch(); const alice = await launch();
const bob = await launch(); const bob = await launch();
const aliceName = `share-a-${tag}`, bobName = `share-b-${tag}`; const aliceName = `share-a-${tag}`, bobName = `share-b-${tag}`;
@@ -243,6 +245,39 @@ try {
assert.equal(await alice.page.locator('.stage .status').count(), 0); assert.equal(await alice.page.locator('.stage .status').count(), 0);
console.log('ok: stopping ends the stream for viewers'); console.log('ok: stopping ends the stream for viewers');
// Camera instead of the screen, and a relay for networks without direct connections: both
// instances point at a mumh5 proxy, and the viewer is limited to the relay over TCP
const relayPort = 19000 + Math.floor(Math.random() * 500);
const relay = spawn(process.execPath, ['server/main.ts'], {
cwd: root, stdio: ['ignore', 'pipe', 'inherit'],
env: { ...env, MUMH5_PORT: String(relayPort), MUMH5_ALLOW_ANY: '1', MUMH5_ALLOW_PRIVATE: '1', MUMH5_STUN_PORT: String(relayPort + 500), MUMH5_STUN_BIND: '127.0.0.1', MUMH5_TURN_IP: '127.0.0.1' }
});
cleanup.push(() => relay.kill());
await new Promise<void>((res, rej) => { relay.stdout.on('data', d => { if (String(d).includes('listening')) res(); }); relay.on('exit', c => rej(new Error(`proxy exited with ${c}`))); });
for (const p of [alice.page, bob.page]) {
await p.getByTitle('Settings').click();
await p.getByLabel('Relay').fill(`http://127.0.0.1:${relayPort}`);
await p.getByLabel('Relay').blur();
await p.getByRole('button', { name: 'Done', exact: true }).click();
}
await bob.page.evaluate(() => localStorage.setItem('mumh5.iceDebug', 'relay-tcp'));
await shareBtn.click();
await picker.getByRole('tab', { name: 'Camera' }).click();
await alice.page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 15000 });
assert.equal(await picker.getByLabel('Sound').count(), 0, 'no extra sound with a camera');
await startBtn.click();
await alice.page.locator('.sidebar .row.user.self').getByTitle('You are sharing your camera').waitFor();
const cam = bob.page.locator('.sidebar').getByRole('button', { name: `Watch the camera of ${aliceName}`, exact: true });
await cam.waitFor();
await cam.click();
await bob.page.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 20000 });
const camStats = await nerd(bob.page, [/^relayed \(TURN\), reached over TCP/, /^0 direct, 0 through STUN, [1-9]\d* relayed/]);
await alice.page.locator('.me').getByRole('button', { name: 'Stop sharing your camera' }).click();
await cam.waitFor({ state: 'detached' });
await bob.page.evaluate(() => localStorage.removeItem('mumh5.iceDebug'));
console.log(`ok: camera shared, watched through the relay set in the desktop app (${camStats.match(/Picture \| [^|]+/)?.[0]})`);
// A sharer who disconnects takes the stream with them // A sharer who disconnects takes the stream with them
await shareBtn.click(); await shareBtn.click();
await pickAndStart(); await pickAndStart();
@@ -260,6 +295,7 @@ try {
console.log('SHARE E2E PASSED'); console.log('SHARE E2E PASSED');
} finally { } finally {
for (const c of cleanup) c();
await alice.app.close().catch(() => {}); await alice.app.close().catch(() => {});
await bob.app.close().catch(() => {}); await bob.app.close().catch(() => {});
} }
+12
View File
@@ -212,6 +212,18 @@ try {
await stats.getByText(/^\d+ kbit\/s$/).first().waitFor({ timeout: 8000 }); await stats.getByText(/^\d+ kbit\/s$/).first().waitFor({ timeout: 8000 });
console.log('ok: screen sharing to a browser without direct UDP, through the relay over TCP'); console.log('ok: screen sharing to a browser without direct UDP, through the relay over TCP');
// A camera instead, still watched through the relay over TCP
await page.locator('.me').getByRole('button', { name: 'Stop sharing your screen' }).click();
await page2.getByRole('region', { name: 'Screen share' }).waitFor({ state: 'detached' });
await page.locator('.me').getByRole('button', { name: 'Share your screen or camera' }).click();
await dialog.getByRole('tab', { name: 'Camera' }).click();
await page.waitForFunction(() => (document.querySelector('.preview video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 });
await dialog.getByRole('button', { name: 'Start sharing' }).click();
await page2.locator('.sidebar').getByRole('button', { name: `Watch the camera of ${name}`, exact: true }).click();
await page2.waitForFunction(() => (document.querySelector('.spot video') as HTMLVideoElement | null)?.videoWidth! > 0, null, { timeout: 30000 });
await page.locator('.stage .status', { hasText: '1 watching' }).waitFor();
console.log('ok: camera shared between two browsers, through the relay');
console.log('WEB E2E PASSED'); console.log('WEB E2E PASSED');
} finally { } finally {
bob.disconnect(); bob.disconnect();
+5 -2
View File
@@ -111,9 +111,12 @@ test('the proxy answers STUN over UDP and announces the port', async () => {
// Relay credentials for people on this site: a username that expires and its password // Relay credentials for people on this site: a username that expires and its password
const base = `http://127.0.0.1:${proxy.port}`; const base = `http://127.0.0.1:${proxy.port}`;
assert.equal((await (await fetch(`${base}/api/config`)).json()).turn, true); assert.equal((await (await fetch(`${base}/api/config`)).json()).turn, true);
const cred = await (await post(base, 'turn', {})).json(); // What clients need for screen sharing: the port, and relay credentials that expire.
// Readable from any origin, since the desktop app asks from outside this site.
const res = await fetch(`${base}/api/relay`, { headers: { Origin: 'app://mumh5' } });
assert.equal(res.headers.get('access-control-allow-origin'), '*');
const cred = await res.json();
assert.ok(Number(cred.username) > Date.now() / 1000 && cred.credential.length > 20 && cred.port === proxy.stunPort); assert.ok(Number(cred.username) > Date.now() / 1000 && cred.credential.length > 20 && cred.port === proxy.stunPort);
assert.equal((await post(base, 'turn', {}, { Origin: 'https://evil.example' })).status, 200, 'origins are open in this test');
assert.deepEqual([...answer.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [127, 0, 0, 1]); assert.deepEqual([...answer.subarray(28)].map((b, i) => b ^ [0x21, 0x12, 0xa4, 0x42][i]), [127, 0, 0, 1]);
client.close(); client.close();
} finally { } finally {