Aug 04, 2025 21:00:00

Kyber, an open source kit for ultra-low latency video streaming developed by the developers of VLC

Jean-Baptiste Kempf, known as JB, the lead developer of

the VideoLAN/VLC project, has announced Kyber , a real-time control SDK designed to reduce latency in video transmission as much as possible. Kyber is built on open source platforms such as FFmpeg and VLC, and is designed for applications that require ultra-low latency, such as cloud gaming, robotics, drones, and remotely controlled vehicles.

Ultra-Low Latency Video Control – An Interview with Jean-Baptiste Kempf of Kyber - Streaming Learning Center
https://streaminglearningcenter.com/codecs/an-interview-with-jean-baptiste-kempf-of-kyber.html

Jean-Baptiste Kempf - Kyber: a new approach for real-time video and controls streaming based on Quic - YouTube


Kyber is not a one-way broadcast, but is intended for use cases where two-way control is essential, such as cloud gaming, remote desktop, remote control of drones and robots, or upstreaming to run a single application on powerful machines in the cloud.

Kyber is not just a library, but a comprehensive solution covering client, server, and network stacks. It not only supports streaming video, audio, and subtitles, but also supports bidirectional streaming of inputs, which is considered difficult to implement. This includes gamepad controls, mouse movements, and even USB command transmission.

Kyber prioritizes minimizing latency, rather than quality or audio-video synchronization, as traditional streaming technologies do. This approach has enabled it to reduce video transmission latency to just a few frames, or tens of milliseconds.

In a lecture given in 2024, a demo video was shown of footage captured from 'Street Fighter 6' being transmitted over Kyber. The training mode of 'Street Fighter 6' has a frame meter display, and comparing the client side (left) and the server side (right) reveals a latency of only about 2 frames, or about 33 milliseconds. In a February 2025 interview, JB reported that they had succeeded in achieving a latency of just 8 milliseconds in a 240Hz environment using 'glass-to-glass' transmission from the camera to the display.

Another key feature of Kyber is that it does not use

WebRTC at all; instead, the server side has been redesigned to run FFmpeg in 'push mode.' At the same time, the VLC-based client side intentionally eliminates audio and video synchronization and buffering, minimizing latency. Kyber runs on a variety of platforms, including Windows, macOS, and Linux, on both the client and server sides, and is highly flexible, supporting all types of hardware and software video codecs.

Kyber uses QUIC and WebTransport to aggregate all data, including video, audio, and control signals, into a single socket for efficient transmission. According to JB, video and audio data are sent as 'datagrams' rather than as standard streams that guarantee reliability.

When packet loss occurs during communication, normal protocols require the lost data to be resent, which causes delays. However, instead of waiting for retransmissions, Kyber uses

a forward error correction technology called ' RaptorQ ' to instantly restore lost packets on the receiving side. While this method slightly increases the bandwidth required for data transmission, it fundamentally eliminates the fatal delays caused by retransmissions, dramatically reducing overall latency.

Furthermore, because Kyber controls the entire protocol itself, it can detect network fluctuations, or increased jitter , and immediately send feedback to the encoder to lower the bitrate, maintaining stable streaming.

Input processing is handled by a dedicated server developed from scratch using Rust . This server not only transmits operations from standard input devices such as keyboards, mice, and gamepads, but also supports advanced features such as clipboard copy and paste, file transfer, and even USB/IP . This allows, for example, a Wacom pen tablet connected to a client PC to be remotely controlled seamlessly and without delay, as if it were directly connected to the server machine.

JB is proud that Kyber's performance is unparalleled, being cutting-edge technology unmatched in both commercial and non-commercial use. Kyber will be released as open source under the AGPL license , which he says is intended to accelerate the spread of the technology. On the other hand, a dual-license model will be adopted, allowing companies to incorporate it into their products under a commercial license, balancing community growth with business sustainability. However, at the time of writing, Kyber's source repository and documentation do not appear to be publicly available.

Regarding the next-generation codec VVC (H.266) , JB expressed skepticism about its widespread adoption, citing its more complex patent issues than HEVC and its niche use cases. However, FFmpeg and VLC support VVC despite his own views, based on their belief that their goal is to 'support everything.' He predicts that in the future, H.264 and AV1 will become mainstream for many applications, while HEVC will continue to be used in the broadcasting field.