0.66.0 - H.264 over WebRTC
Until now the picture came over a WebSocket, which runs on TCP. TCP delivers everything in order, so one lost packet holds up everything behind it until it is sent again. On a bad Wi-Fi or mobile link that is a picture that stops for a second or two.
In 0.66.0 the console takes H.264 over WebRTC when it can. WebRTC runs on UDP: a lost packet costs one frame, and the browser asks for it again if there is time. Keys, the mouse and everything else stay on the WebSocket. If WebRTC does not connect, the console uses the WebSocket as before.
The WebRTC part on the device is my own small one: ICE-lite, DTLS-SRTP, H.264 in RTP, resends and keyframe requests. The device uses no STUN or TURN server. It lists its own addresses, Tailscale included, and the browser picks one. Settings -> Video -> Allow WebRTC turns it off, and the Video readout says which way the picture comes.
The numbers
Mouse to picture, measured with the console's "Measure the delay":
| Path | WebSocket | WebRTC |
|---|---|---|
| PC on the LAN | 166 ms | 133 ms |
| PC over Tailscale, direct | 170 ms | 138 ms |
| Phone on mobile data, Tailscale | - | 186 ms |
WebRTC is about 30 ms faster even on a clean link. Part of that is one header: the device asks Chrome to show each frame at once, without its smoothing buffer (the playout-delay RTP extension). On the phone that took the delay from 315 to 186 ms. The frame rate does not change with the transport: about 23 fps at 1080p either way, which is what the encoder gives.
WebSocket over Tailscale
The WebSocket over Tailscale, in Chrome on a PC:
| 0.65.0 | 0.66.0 | |
|---|---|---|
| Mouse to picture | 2266 ms | 170 ms |
| Frames | 2 every 2.5 s | 23 fps |
The cause was a bug in the firmware, in the code that sends video, not in Tailscale. After a keyframe that code skipped the next frame, because lwIP calls a socket writable only when half its send buffer is free, and over a VPN the keyframe had not drained by then. A skipped H.264 frame needs a new keyframe, so it went on: one picture every 2.5 s. A viewer that acks frames is now paced by its acks alone.
Also in 0.66.0
- Tailscale through a relay: Nagle is off there too, a packet goes as one TLS record instead of two, and the queue holds 64 packets instead of 16.
- The Video popup has a copy button and shows the browser's buffer, lost packets and both round trips.
- The on-screen keyboard showed capitals all the time. Letters now follow Shift and the target's Caps Lock.