Add Video backend for Linux

[mafiesto4]

Add missing video backend for Linux platform. Find a good lib or 3rd party plugin for .mp4 files loading (eg. minimp4), then decode h.254 video frames into YUV format (engine can automatically convert into RGB image). Use one of the existing backends as base for proper implementation: https://github.com/FlaxEngine/FlaxEngine/tree/1.9/Source/Engine/Video

All other platforms provide system-lib for video media playback so only Linux is missing.

Video docs: https://github.com/FlaxEngine/FlaxDocs/blob/1.9/manual/graphics/video/index.md

Here is a prototype with minimp4 that loads data from a file but requires frames decoding: PrototypeMinimp4.zip

[Zode]

https://en.wikipedia.org/wiki/Advanced_Video_Coding#Licensing Due to the licensing nature of mp4/h264, linux may need the use of webm/vp8 or equivalent. Thoughts on using https://gstreamer.freedesktop.org/ to do the video decoding job? It is the most widely used in apps, and is what you would get if games were ran through proton/wine. Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

[mafiesto4]

Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

I think it would be a good starting point. Maybe we could use ffmpeg?

[Zode]

Could be dynamically linked so that it uses system binaries for the job, so it can use "nonfree" decoders (such as nvidia's gpu decoding).

I think it would be a good starting point. Maybe we could use ffmpeg?

FFmpeg is bit harder as it provides a low level API from what i gather, GStreamer provides a high level API similar to the MFPlay API used on Windows side of things so it would be way simpler to integrate (and more importantly: maintain)

edit: FFmpeg is the way, GStreamer has issues with vulkan (or rather lack of feature support) 😩

[alsed]

I have made a simple mp4 player, trying to solve this problem. I just uploaded to github:

https://github.com/alsed/simple-mp4-player

It uses minimp4 as demuxer, openh264 as decoder, (and SDL2 to play the frames since it takes directly the YUV format). The first part of the code is a copy of minimp4, then comes the decoding, where the hardest part was to use correctly the stride data. The decoder is slow, but optimizations can be made. I left the code simple on purpose. I hope you find it useful @mafiesto4