ingress icon indicating copy to clipboard operation
ingress copied to clipboard

WHIP Ingress with Simulcast (bypass transcoding)

Open mpisat opened this issue 8 months ago • 0 comments

There is a project to bring Simulcast publishing to OBS Whip by Sean-Der - https://github.com/obsproject/obs-studio/pull/10885 Currently it works with his Glimesh/broadcast-box and OvenMediaEngine (both opensource projects). You may ask why use Ingress when livekit can ingest Simulcast with a web browser. Well, OBS can produce much higher bitrates and encode video at higher quality compared to a web browser and some people simply prefer OBS over Web based publishing.

When I try simulcast publishing with 3 layers, to an ingress that has bypass transcoding, it can see the tracks but track switching is unavailable.

lk room participants get --room whip whip
Using url, api-key, api-secret from environment
Using url, api-key, api-secret from environment
{
  "sid": "PA_uwLQg9Esga8z",
  "identity": "whip",
  "state": 2,
  "tracks": [
    {
      "sid": "TR_AMB6CHz2snxRrZ",
      "source": 2,
      "mime_type": "audio/opus",
      "mid": "1",
      "stereo": true,
      "stream": "camera",
      "version": {
        "unix_micro": 1747025474889633
      },
      "audio_features": [
        0
      ]
    },
    {
      "sid": "TR_VCzNxnGMMBe75H",
      "type": 1,
      "width": 1920,
      "height": 1080,
      "simulcast": true,
      "source": 1,
      "layers": [
        {
          "quality": 2,
          "width": 640,
          "height": 360,
          "ssrc": 3072329210
        },
        {
          "quality": 1,
          "width": 1280,
          "height": 720,
          "ssrc": 745007582
        },
        {
          "width": 1920,
          "height": 1080,
          "ssrc": 421479776
        }
      ],
      "mime_type": "video/H264",
      "mid": "2",
      "codecs": [
        {
          "mime_type": "video/H264",
          "mid": "2",
          "cid": "IN_qEUWGi5Mp8XM",
          "layers": [
            {
              "quality": 2,
              "width": 640,
              "height": 360,
              "ssrc": 3072329210
            },
            {
              "quality": 1,
              "width": 1280,
              "height": 720,
              "ssrc": 745007582
            },
            {
              "width": 1920,
              "height": 1080,
              "ssrc": 421479776
            }
          ]
        }
      ],
      "stream": "camera",
      "version": {
        "unix_micro": 1747025475059295
      }
    }
  ],
  "joined_at": 1747025474,
  "joined_at_ms": 1747025474813,
  "name": "whip",
  "version": 7,
  "permission": {
    "can_publish": true,
    "can_publish_data": true,
    "can_update_metadata": true
  },
  "region": "de-berlin",
  "is_publisher": true,
  "kind": 1,
  "attributes": {
    "ingress.ingressID": "IN_qEUWGi5Mp8XM",
    "ingress.resourceID": "WH_yWf8R4Amr8Nz"
  }
}

When I try it to an ingress that has WHIP with transcoding enabled, it refuses it, which is okay.

2025-05-12T06:27:58.868+0200    INFO    ingress service/service.go:316  received ingress info   {"nodeID": "NE_t5RhELR7WZep", "ingressID": "IN_tF4cW97aR59n", "streamKey": "h3vxyWasTHCC", "resourceID": "WH_qKmDeLfC3pT6", "ingressInfo": "ingress_id:\"IN_tF4cW97aR59n\"  name:\"whiptranscoding\"  stream_key:\"{h3v...HCC}\"  url:\"http://xx.xxx.com:8038/whip\"  input_type:WHIP_INPUT  enable_transcoding:true  audio:{source:MICROPHONE  preset:OPUS_STEREO_96KBPS}  video:{source:CAMERA  preset:H264_1080P_30FPS_3_LAYERS}  room_name:\"whiptranscoding\"  participant_identity:\"whiptranscoding\"  participant_name:\"whiptranscoding\"  reusable:true  state:{status:ENDPOINT_BUFFERING  started_at:1747024078864848760  updated_at:1747024078865295343  resource_id:\"WH_qKmDeLfC3pT6\"}"}
2025-05-12T06:27:58.868+0200    WARN    ingress service/service.go:174  ingress failed  {"nodeID": "NE_t5RhELR7WZep", "error": "simulcast is not supported when transcoding"}

Considering you allow Simulcast on WHIP bypass transcoding, is there any plan to get it working? PS: high, medium, low mapping in simulcast (whip - bypass) is in reverse order.

mpisat avatar May 12 '25 04:05 mpisat