# SRT to WHEP

This application ingests one MPEG-TS over SRT stream and outputs to WebRTC recvonly clients using WHEP as signaling protocol. Example of use cases:

- Browser based confidence monitor of an incoming stream
- Program or preview output monitor in a browser or tablet

Supports SRT streams in caller and listener mode.
Runs on MacOS and Ubuntu.

---
<div align="center">

## Quick Demo: Open Source Cloud

Run this service in the cloud with a single click.

[![Badge OSC](https://img.shields.io/badge/Try%20it%20out!-1E3A8A?style=for-the-badge&logo=data:image/svg+xml;base64, )](https://app.osaas.io/browse/eyevinn-srt-whep?utm_source=github&utm_medium=readme&utm_campaign=srt-whep)

</div>

---

![screenshot](docs/screenshot.png)

## Architecture

Internally the flow is: SRT ingest → a signaling coordinator (a single actor
that owns all connection state) → one hot-plugged GStreamer branch per viewer
→ WebRTC/WHEP out. Each viewer's branch reaches the signaling plane through an
in-process loopback WHIP bridge, so the pipeline never imports signaling code.

![Animated architecture diagram: SRT source feeds the GStreamer pipeline; the
coordinator actor owns connection state and drives branch add/remove; HTTP
routes talk to it via SignalHandle; the connection lifecycle advances
AwaitingOffer → AwaitingAnswer → Established, with the sweep reaping timeouts
and the watchdog escalating to a supervisor restart](docs/srt-whep-coordinator-actor.gif)

For the module map and domain glossary see [`CONTEXT.md`](./CONTEXT.md); the
design decisions behind this shape are recorded in [`docs/adr/`](./docs/adr/).

## Design Principles
When conceiving this project, we made deliberate design choices to shape its functionality and behavior in alignment with our vision:

- No Video Transcoding by Default:
We have intentionally refrained from transcoding video in the default configuration. This decision stems from the desire to maintain the integrity of the original video stream. This approach caters to use cases like confidence monitoring or preview. If you wish to transcode video anyway, run with the `-D`/`--decode-video` option: the incoming H264 stream is then decoded and re-encoded by the WebRTC sink for each viewer.
- Server-Side Initiation Mode:
Our project does not currently support the client-side initiation mode of [WebRTC-HTTP Egress Protocol (WHEP)](./docs/whep.md). Instead, we have adopted a server-side initiation approach. In this mode, the server provides the SDP offer, as it possesses knowledge of the available streams.
- Focus on SDP Offer/Answer Exchange:
Our server's primary focus is on the exchange of SDP (Session Description Protocol) offers and answers. While WebRTC typically involves ICE (Interactive Connectivity Establishment) negotiation for establishing peer-to-peer connections, we have opted not to include ICE negotiation within the scope of our project at this time. Our emphasis remains on the effective exchange of SDP-related information.

## Compliance Table
Here we have a list of supported production software.

| Source Type       | Supported            | Details                                  |
|-------------------|----------------------|------------------------------------------|
| FFMpeg            | :white_check_mark:   | A complete, cross-platform solution to record, convert and stream audio and video |
| [OBS](https://obsproject.com)               | :white_check_mark:   | Free and open source software for video recording and live streaming                 |
| GStreamer         | :white_check_mark:   | A pipeline-based multimedia framework    |
| [Agile Live](https://docs.agilecontent.com/docs/acl/) | :white_check_mark: | A software-based platform for production of media using standard, off-the-shelf hardware and datacenter building blocks. |
| [Intinor Direkt](https://intinor.com/) | :white_check_mark: | Intinor Direkt series is a product line of encoders and decoders for live video over the internet. |

## Getting Started
It is suggested that

- Mac Users: Follow our comprehensive build instructions and use Chrome for streaming.
- Ubuntu Users: Either build the program from source or use Docker for running. Both methods are supported.

Some useful commands can be found [here](./docs/useful_commands.md).

## Install

```
cargo install srt_whep

# recommended for pretty log viewer (optional)
cargo install bunyan
```

Generate an SRT test source for example using our testsrc Docker container:

```
docker run --rm -p 1234:1234/udp eyevinntechnology/testsrc
```

An SRT stream (in listener mode) is then available at `srt://127.0.0.1:1234`. Then run the `srt-whep` application:

```
srt-whep -i 127.0.0.1:1234 -o 127.0.0.1:8888 -p 8000 -s caller | bunyan
```

It will connect to the SRT test stream in caller mode as the generated SRT stream is in listener mode.

WHEP endpoint is available at `http://localhost:8000/channel`. You can then play it for example using the WHEP [Player](https://webrtc.player.eyevinn.technology/?type=whep). Possible issues are discussed in [Issues](#issues).

If you don't have Rust install you can use the Docker Container image published on Docker Hub:

```
docker run --rm --network host eyevinntechnology/srt-whep \
  -i 127.0.0.1:1234 \
  -o 0.0.0.0:8888 \
  -p 8000 -s caller
```

Note that the container needs to run in host-mode (supported only on Linux).

## Build from Source
### OSX

Requirements:
- XCode command line tools installed
- GStreamer [binaries](https://gstreamer.freedesktop.org/data/pkg/osx/) from GStreamer's website installed — a recent version that **bundles the `rswebrtc` plugin** (it provides `whipclientsink`; verify with `gst-inspect-1.0 whipclientsink`). srt-whep uses whichever `rswebrtc` the installation provides rather than compiling its own copy in (see [`docs/adr/0003`](./docs/adr/0003-webrtc-plugin-from-installation.md)).
- Rust and cargo installed

Make sure you have the following env variables defined:

```
export PATH=$PATH:/Library/Frameworks/GStreamer.framework/Versions/Current/bin
export PKG_CONFIG_PATH=/Library/Frameworks/GStreamer.framework/Versions/Current/lib/pkgconfig
export GST_PLUGIN_PATH=/Library/Frameworks/GStreamer.framework/Versions/Current/lib
export DYLD_FALLBACK_LIBRARY_PATH=$GST_PLUGIN_PATH
```

> Keep `GST_PLUGIN_PATH` pointed at the GStreamer installation only. Do **not**
> prepend a separately-built `gst-plugins-rs`: if it ships a higher-versioned
> `rswebrtc` it wins the plugin registry and replaces the installation's
> matched one, which can break the WebRTC media path (see
> [`docs/adr/0003`](./docs/adr/0003-webrtc-plugin-from-installation.md)).

Build with Cargo

```
cargo check
cargo install bunyan # Optional, for pretty printing of logs
cargo build --release
```

The binary is then available at `./target/release/srt-whep`. See below for how to run it.

### macOS Workaround: H264 Video Not Rendering in Browser

Due to a caps negotiation bug in `webrtcsink` 0.15.0 on macOS, H264 passthrough fails with a `not-negotiated` error on `GstAppSrc:video_0` after a few seconds. The workaround is the `-D` / `--decode-video` flag, which inserts an `avdec_h264` decode step before `whipclientsink`. This causes webrtcsink to receive raw video and re-encode internally (typically as VP8), bypassing the passthrough bug.

```
export GST_PLUGIN_FEATURE_RANK="vtenc_h264:NONE,vtenc_h264_hw:NONE"
./target/release/srt-whep -i 127.0.0.1:1234 -p 8000 -D
```

The `GST_PLUGIN_FEATURE_RANK` env var disables Apple's hardware H264 encoders (`vtenc_h264`, `vtenc_h264_hw`), which also fail with not-negotiated errors when webrtcsink attempts to use them for re-encoding.

> Note: The `-D` flag introduces a decode+re-encode step and is not intended for production use. It is a local development workaround until the upstream webrtcsink bug is resolved.

### Debian (bullseye / bookworm)

Requirements:
- Rust and cargo installed

Install GStreamer build dependencies.

```
apt-get update
apt-get -y install build-essential \
  curl \
  pkg-config \
  libssl-dev \
  libunwind-dev \
  libgstreamer1.0-dev \
  libgstreamer-plugins-base1.0-dev \
  libgstreamer-plugins-bad1.0-dev \
  gstreamer1.0-plugins-base \
  gstreamer1.0-plugins-good \
  gstreamer1.0-plugins-bad \
  gstreamer1.0-plugins-ugly \
  gstreamer1.0-libav \
  gstreamer1.0-tools \
  gstreamer1.0-x \
  gstreamer1.0-alsa \
  gstreamer1.0-gl \
  gstreamer1.0-gtk3 \
  gstreamer1.0-qt5 \
  gstreamer1.0-pulseaudio \
  gstreamer1.0-nice
```

> **WebRTC output needs the `rswebrtc` plugin, which the packages above do
> not include.** srt-whep loads `whipclientsink` from the installed
> `rswebrtc`/`gst-plugins-rs` plugin rather than compiling its own copy (see
> [`docs/adr/0003`](./docs/adr/0003-webrtc-plugin-from-installation.md)), and
> no Debian/Ubuntu apt package ships it. Without it the app ingests SRT but
> every WHEP viewer fails. Build [`gst-plugins-rs`](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs)
> and put `libgstrswebrtc.so` on `GST_PLUGIN_PATH`, or just use the Docker
> image, whose base already bundles it (see [`docs/adr/0004`](./docs/adr/0004-docker-runtime-base-with-rswebrtc.md)).
> Verify with `gst-inspect-1.0 whipclientsink`.

Build with Cargo

```
cargo check
cargo install bunyan # Optional, for pretty printing of logs
cargo build --release
```

The binary is then available at `./target/release/srt-whep`. See below for how to run it.

## Docker Container

Build container (uses multi-stage builds):

```
docker build -t srt-whep:dev .
```

The runtime image is based on [`livekit/gstreamer`](https://hub.docker.com/r/livekit/gstreamer)'s
`-prod-rs` tag because it bundles the `rswebrtc` plugin that provides
`whipclientsink` (the WebRTC output element). No Debian/Ubuntu apt package ships
that plugin, so a stock `gstreamer1.0-plugins-*` install alone is not enough —
see [`docs/adr/0004`](./docs/adr/0004-docker-runtime-base-with-rswebrtc.md).

Container must be running in host-mode (only works on Linux hosts, and is not supported on Docker Desktop for Mac, Docker Desktop for Windows)

```
docker run --rm --network host srt-whep:dev \
  -i <SRT_SOURCE_IP>:<SRT_SOURCE_PORT> \
  -o 0.0.0.0:8888 \
  -p 8000 -s caller
```

## Usage

To ingest an SRT stream with address `srt://127.0.0.1:1234` in listener mode and expose WHEP endpoint on port 8000 run the application with this command.

```
cargo run --release -- -i 127.0.0.1:1234 -o 127.0.0.1:8888 -p 8000 -s caller | bunyan
```

This will also make a pass-through of the SRT stream on `srt://127.0.0.1:8888` in listener mode. To watch the pass-through stream in ffplay, VLC or GStreamer you run:

```
ffplay srt://127.0.0.1:8888
# or
gst-launch-1.0 playbin uri="srt://127.0.0.1:8888"
```

WHEP endpoint is available then at `http://localhost:8000/channel`. You can then play it for example using the WHEP [Player](https://webrtc.player.eyevinn.technology/?type=whep).

If the SRT stream to ingest is in caller mode you run the application with this command.

```
cargo run --release -- -i 127.0.0.1:1234 -o 127.0.0.1:8888 -p 8000 -s listener | bunyan
```

This also expects the SRT address `127.0.0.1:8888` to be running in caller mode.

## Testing

To verify **end-to-end that a real browser actually decodes the media** — not just that the WHEP connection is established — run the automated browser check (macOS):

```
tests/browser/run.sh                 # headless Chrome, default profile
tests/browser/run.sh --headed        # watch the Chrome window
```

It brings up an SRT test source and srt-whep, serves a diagnostic WHEP player, drives the real system Google Chrome through the full WHEP handshake, and exits `0`/`1` based on whether video frames actually decode — so it catches the "connected but zero media" failure that a plain connection check misses. It prints a per-stage report and writes JSON to `target/codec-test/`. See [`tests/browser/README.md`](./tests/browser/README.md) for prerequisites and options (profiles, `--headed`, `--skip-bringup`). For the H.264 profile × browser compatibility matrix (including Safari), see [`docs/supported_codecs.md`](./docs/supported_codecs.md).

The Rust unit and integration tests run with:

```
cargo test
```

## Tips for Successful Streaming

When working with SRT streams, there are several important considerations that can affect the success of your streaming experience:

1. **Stream Generation and Playback Tools:**
- Different tools, such as FFMpeg, GStreamer, FFPlay, and VLC, can be used to generate and play SRT streams. We've tested our program with these tools, so you can choose the one that suits your needs.
- Be aware of the mode option in the SRT stream configuration, which can be set as `caller`, `listener`, or `rendezvous`. This option determines the behavior of the stream, and it needs to be configured correctly for successful streaming.

2. **Bitrate Parameter in FFMpeg Streams:**
- When generating streams using FFMpeg, it's essential to specify the Bitrate parameter. Failing to do so might result in VLC/GStreamer being unable to play the stream.

3. **Video Codecs and Profiles:**
- WebRTC connection failures can often be attributed to incompatible video codecs. While we strive to support both H.264 (AVC) and H.265 (HEVC) streams, it's important to note that most mainstream browsers only support AVC for WebRTC.
- Safari, which is the only browser supporting H.265, has its own RTP payload format and custom video profile requirements, different from the standard ([RFC 7798](https://www.rfc-editor.org/rfc/rfc7798.html)). So it does not work directly out of box. There is a [PR](https://github.com/WebKit/WebKit/pull/15494) available for fixing this H265 packetization issue in WebKit project but no one is viewing it for the moment.
- Based on the [discussion](https://github.com/AlexxIT/Blog/issues/5), it seems rather challenging to support H.265 / HEVC right now.

4. **Codecs and Profiles Compatibility:**
- Video profiles, such as Baseline, Main, and High for H.264, play a crucial role in stream compatibility. While Chrome supports all profiles, Safari only accepts the Baseline profile. For further details, please refer to [this](./docs/supported_codecs.md) table.

5. **SRT Stream Latency:**
- SRT streams can be configured to have different latencies. For example, if you're using GStreamer to generate SRT streams, you can set the latency parameter of `srtsink` or `srtsrc` to fit your need. For example, `latency=200` sets the latency to 200ms, which is appropriate for most use cases.
- Our tool adds a minimun latency to the stream by default. If you want to measure the end-to-end latency, you can try to generate a stream using OBS with a clock overlay (which adds a very low latency). Then you can compare the clock in the stream with the built-in clock from our WHEP player. The end-to-end latency should be around 300~500 ms.

6. **Keyframe Interval (GOP) for Late-Joining Viewers:**
- A WebRTC viewer's branch is hot-plugged into the running pipeline, so it starts receiving mid-stream. A decoder needs an IDR keyframe (with its SPS/PPS) before it can render video, and srt-whep does not transcode, so it cannot force the source to emit one — it can only forward the next keyframe the source sends.
- Configure the source with a short keyframe interval so viewers get video quickly. In `x264enc` set `key-int-max=30` (≈1s at 30fps); in FFmpeg use `-g 30`. With a long/infinite GOP, audio (Opus) plays immediately while video stays black until the next keyframe arrives.

## Discussion and Issues
All relevant discussions are tracked in [issues](https://github.com/Eyevinn/srt-whep/issues/). Please feel free to open a new issue if you have any questions.

- We have a set of known limitations recorded [here](./docs/known_limitations.md). You might want to check it out if running into an issue.
- If you doubt a plugin is missing in gstreamer you can check it using `gst-inspect-1.0 <plugin>`. For example, `gst-inspect-1.0 srtsink`.
- To get more verbose logging you can set the `GST_DEBUG` environment variable to `2`. For example, run in terminal: `export GST_DEBUG=2`

## License (Apache-2.0)

Copyright 2023 Eyevinn Technology AB

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

## Support

Join our [community on Slack](http://slack.streamingtech.se) where you can post any questions regarding any of our open source projects. Eyevinn's consulting business can also offer you:

- Further development of this component
- Customization and integration of this component into your platform
- Support and maintenance agreement

Contact [sales@eyevinn.se](mailto:sales@eyevinn.se) if you are interested.

## About Eyevinn Technology

Eyevinn Technology is an independent consultant firm specialized in video and streaming. Independent in a way that we are not commercially tied to any platform or technology vendor.

At Eyevinn, every software developer consultant has a dedicated budget reserved for open source development and contribution to the open source community. This give us room for innovation, team building and personal competence development. And also gives us as a company a way to contribute back to the open source community.

Want to know more about Eyevinn and how it is to work here. Contact us at work@eyevinn.se!
