RenderHeads
iOS mediaPlayer.TextureProducer.GetTexture(0) is returning plain black texture at times after Auto Loop restarts playback.
Describe the issue
We have recently upgraded from 1.x to 2.3.3 and are now having this issue.
On scene load we setup media player and load a HLS video stream, in this case a 10 sec clip.
The clip is opened with mediaPlayer.OpenMedia() and mediaPlayer.Control.SetLooping(true);.
Since it is adaptive bitrate content we setup a function to map the textures to our materials on the following events
case MediaPlayerEvent.EventType.FirstFrameReady:
case MediaPlayerEvent.EventType.ResolutionChanged:
case MediaPlayerEvent.EventType.Unstalled:
The main thing this function does is to map the Texture received from mediaPlayer.TextureProducer.GetTexture(0); to our materials and enable/disable shader keywords. This utilises a WaitUntil to wait until the texture being returned is != null, however from debug logging I can see its not null and the app flow continues past this point regardless of whether the issue is occurring or not so the method looks to be returning a valid texture.
This works fine on Android, Editor and Desktop - however on iOS when the video content ends (10 secs) and auto loops back to the start, we received a ResolutionChanged event which triggers a remapping of the textures. (This is the only event that fires when the video restarts , is this the expected behavior?)
It seems like 30-40% of the time this succeeds, the rest it fails - resulting in our video material being plain black.
There are no errors, just black texture. If we wait 10 secs until the next loop, it may or may not start working as expected - only happening on iOS. I have tested on Mac in Editor and it didn't seem to happen often although I may have seen it once or twice.
Your Setup (please complete the following information):
- Unity version: 2019.4.33f1
- AVPro Video version (number and edition (trial/core/ultra/enterprise)): AVPro Core 2.3.3
- Operating system version: iOS 15.2
- Device model: iPhone SE 2020
- Video specs (resolution, frame-rate, codec, file size): Source is 3840x2160, 30FPS, 10Mbps, H264 / AAC LC. Output is HLS , see URL below.
To Reproduce
- OpenMedia , set to URI : https://stage-vod.akamaized.net/hls/gm_video_squashed169/playlist.m3u8
- SetLooping(true)
- Upon receiving ResolutionChanged event, remap the materials texture to mediaPlayer.TextureProducer.GetTexture(0) - observe plain black texture rendering, no errors, 60-70% of the time (after each 10 second loop)
Logs There are no errors being reported in the console logs.
Screenshots
Videos https://stage-vod.akamaized.net/hls/gm_video_squashed169/playlist.m3u8
I've reproduced this. Problem is the texture update lags behind the updated info from the stream. Will look into getting this fixed. A work around for now would be to delay your texture update to the end of the next frame.
Great thanks for this, will test out your suggestion for now and I'll keep an eye out for a fix. Thanks for the prompt response.
I tried a few variations of skipping a frame and then waiting until the end of frame etc but it doesn't seem to completely fix it unfortunately. That said it does seem like its quite consistently working each 2nd playback now. E.g plays correct for 10 secs, then black for 10 sec, then correct etc. Previously it was black much more often than rendering correctly.
I guess you're using your own scripts to update the material as I was unable to reproduce this using our ApplyToMaterial.cs script. It checks to see if the texture from TextureProducer.GetTexture(0) has changed every frame and if it has, remaps it. This might work better than waiting for an event.
I implemented the same technique used in your ApplyToMaterial script (we based our original implementation off this but an old 1.x version) and I'm still seeing the same issue on iOS only. The only way I can make it work is to disable auto looping and implement our own crude manual loop process (e.g upon FinishedPlaying event, if a loop bool is true, re-open the media and play again if the content is supposed to loop. I assume this probably carries more overhead but it does the job as a workaround.
We use multiple player items internally to get seamless looping to work and for some reason when it switches the player item the texture update isn't happening. I'm still trying to get a 100% reproducible instance here.
There is an option in the player settings Use single player item which when enabled will only use one player item and will perform a seek back to the beginning when looping, it might work better than your workaround.
If you're willing, could you try replacing AppleMediaPlayer.cs (located at Assets/AVProVideo/Runtime/Scripts/Internal/Players/) with the one in the attached archive.
This removes Texture2D recreation when the resolution changes and therefore you won't need to remap your materials. The underlying texture will still behave correctly but Texture2D.width and .height will no longer be correct after a resolution change and will report the original resolution. MediaPlayer.Info.GetVideoWidth() and MediaPlayer.Info.GetVideoHeight() will still be correct though.
