Stream Freezes and Skips Randomly on Home Assistant Companion App

[thinkjk]

On my local network on the Android and iOS home assistant companion app my stream freezes after a few seconds and I can’t get it to unfreeze without navigating away on the app. Sometimes it’ll freeze, then jump forward a bunch of seconds and freeze again. It works perfectly in my browser.

It only happens with WebRTC. RTMP, RTSP, and JSMpeg play without any issues.

Here’s my current config:

Camera → RTSP → Frigate → RTMP → Home Assistant. I have RTSPtoWebRTC pointing to go2rtc. My go2rtc.yaml is empty. I see the RTMP stream added automatically in the go2rtc UI when I start a stream.

I see the freezing issue with the Frigate Lovelace Card as well as the Picture Glace card.

Troubleshooting I’ve done: If I remove go2rtc and RTSPtoWebRTC and add a generic camera with a direct RTSP from the camera it does not freeze. If I add the RTMP stream from Frigate the stream does no freeze. I tried using your older WebRTC plugin only and I get the same issue, stream freezes when it flips to WebRTC (I know it’s webrtc because I’ve verified in developer tools and the icon flipped to webrtc in the corner)

Additional testing. I have 3 types of Amcrest cameras and it seems like the issue is mainly happening on one model the ip4m-1051. Is there anything I can do to troubleshoot or resolve the issue?

Thanks.

[AlexxIT]

You can share your camera stream with me, so I can check it with debugger. You can do it using Ngrok tunnel as example.

[thinkjk]

Sure! Thanks @AlexxIT. Can you quickly let me know how I can test the stream externally? I have the ngrok client installed on home assistant from https://github.com/dy1io/hassio-addons. I added the ngrok.yaml and updated my go2rtc.yaml to point to the config (per your docs). I opened up the WebRTC and API port on my firewall, but I'm not sure how to test if I can connect externally.

Edit: I get an error in the go2rtc logs saying I'm limited to one connection ERR_NGROK_108. I think this is because I was having the client from the add-on and go2rtc try to make a connection. I removed the api key from the add-on while keeping it only in the ngrok.yaml file so go2rtc can use it. I still can't figure out how to test the stream externally.

Can I have a friend try to hit the stream somehow with VLC?

[AlexxIT]

https://github.com/AlexxIT/go2rtc/wiki/Tunnel-RTSP-camera-to-Intenet

You can send link to contacts from my github page

[thinkjk]

@AlexxIT I sent you a friend request on Discord unless you prefer Telegram (which I've never used)

[thinkjk]

Leaving a final note. Thanks to @AlexxIT for the troubleshooting help. This was due to 2 issues.

1. The ip4m-1051 streams in very very high quality, so lower the quality helped with the issue a bit (though it was still dropping frames.).
2. The fix ended up being that there was added latency coming from Frigate (RTMP). I had to add a generic camera (RTSP) and then I could stream without issues or lag.

[rogerquake]

After I switched to the direct RTSP feeds from my cameras and not the Frigate RTMP feeds, I am not seeing any freezing issues either.

[thinkjk]

@AlexxIT this issue started happening again on the newer versions of the app. @rogerquake are you running into this as well running the newest version?

[rogerquake]

Yeah, my streams have frozen twice on the latest release.

[AlexxIT]

Well. Beta9 has this problem, but Beta8 doesn't?

[rogerquake]

I don't think it's any different honestly. Beta8 froze also, so don't think Beta9 is any worse.

[AlexxIT]

Well. Which version was definitely OK?

[thinkjk]

Beta5 was my last confirmed working version for me. I unfortunately don't have a backup of that version but I just tested Beta6 and get the same freezing issue. There were also updated with the home assistant mobile app since that time.

[NickM-27]

Beta5 was my last confirmed working version for me. I unfortunately don't have a backup of that version but I just tested Beta6 and get the same freezing issue. There were also updated with the home assistant mobile app since that time.

You can load previous releases in the releases section. https://github.com/AlexxIT/go2rtc/releases/tag/v0.1-beta.5

[AlexxIT]

If you put any go2rtc binary to Hass config folder - it will be used by the add-on. Don't forget to set chmod +x

[thinkjk]

I just tried this and it's still showing beta9 in the UI. I have go2rtc_linux_amd64 in my config folder and ran a chmod +x on it and rebooted HA.

[AlexxIT]

Binary should be named just go2rtc. Version in the UI won't change.

[thinkjk]

If you put any go2rtc binary to Hass config folder - it will be used by the add-on. Don't forget to set chmod +x

Thanks @AlexxIT. I just confirmed that with beta5 I don't have any issues at all, but every version after that my stream freezes and jumps 10+ seconds at a time. I tested with 6-9

[AlexxIT]

I need to access your stream again.

[NickM-27]

I'm seeing this with my streams while viewing webRTC in the browser. At least in my case it seems only the video stutters. I can hear up to date audio even when the frame for video is frozen.

[AlexxIT]

It's important to provide full info about source of your stream. Technology, transcoding, brands.

[NickM-27]

I've seen this on my amcrest ad410 which is h.264 / pcma. This one uses the direct rtsp link from the camera

I've also seen this on my reolink 511-WA using their http:// link with ffmpeg: preset. The stream is native h.264 and aac and I am using #video=copy#audio=opus to transcode the audio so it is usable.

[NickM-27]

I realized maybe this is more of a client issue? Watching with webrtc on my windows or mac desktop in chrome the stream does not skip at all.

However, watching in chrome on my android does see the stuttering quite often. Watching in chrome on my iPhone does not see the suttering.

[thinkjk]

I realized maybe this is more of a client issue? Watching with webrtc on my windows or mac desktop in chrome the stream does not skip at all.

However, watching in chrome on my android does see the stuttering quite often. Watching in chrome on my iPhone does not see the suttering.

If you read my original issue it's the same for me. I have 0 issues on desktop in any browser (Chrome, Firefox, Safari) and OS (Windows, OSX, Ubuntu). My issue only happens on mobile (I'm using the HA Companion App)

[NickM-27]

Okay thanks, glad we're seeing the same thing.

I wonder if there's some chrome flag or other thing that could improve it on Android. My phone has a Snapdragon 8+ Gen 1 so I don't think it's a performance issue.

[AlexxIT]

@NickM-27 can you check SDP for both cameras? You can increase log level to rtsp: trace or use ffplay -v trace from PC. This line:

a=fmtp:96 packetization-mode=1;profile-level-id=420032;sprop-parameter-sets=...

[thinkjk]

Okay thanks, glad we're seeing the same thing.

I wonder if there's some chrome flag or other thing that could improve it on Android. My phone has a Snapdragon 8+ Gen 1 so I don't think it's a performance issue.

Also if you have a minute do you mind trying to see if beta5 gives you the same issues on mobile? That version gives me no problems. @AlexxIT today my stream was freezing again on beta9 but works fine on beta5.

[NickM-27]

@AlexxIT

Amcrest AD410: a=fmtp:96 packetization-mode=1;profile-level-id=420032;sprop-parameter-sets=Z0IAMukAUAHjQgAAB9IAAOqcCAA=,aMqPIAA=

Reolink 511-WA: a=fmtp:96 packetization-mode=1; sprop-parameter-sets=J2QAKq0AzoB4AiflmoCAgPgAAAMACAAAAwDxgYABDzwADLbv/+BQ,KO48sA==; profile-level-id=64002A

[NickM-27]

@thinkjk I just tried beta 5 and I see the stuttering there as well.

[AlexxIT]

@thinkjk try beta.10

[thinkjk]

@thinkjk try beta.10

Thank you, it's much better than before! But I now see this weird issue where it shows the first frame of the video but takes 10-30+ seconds before it actually starts showing the live feed. It takes longer externally than internally. I currently have this in my go2rtc.yaml. Is there something else I should be changing for better connection?

webrtc:
  listen: ":8555"  # address of your local server (TCP)
  candidates:
    - MYIP:8555  # if you have static public IP-address
    ```