FFmpeg.AutoGen and static libraries (iOS)

[digivod]

For a video processing Maui app, I need ffmpeg access on all Maui platforms (ios,android,mac catalyst, windows). iOS enforces static libraries (thanks, Apple...). So I wondered how FFmpeg.AutoGen goodness is possible in this scenario.

Binding to a static ffmpeg library imho effectively means that you don't want this as a public nuget package, because of the different ffmpeg compilations each scenario might require (gpl, lgpl, nonfree, max 3rd party libs vs. minimum...).

So I thought what about an interface for the ffmpeg calls. S.th. like this:

public interface Iffmpeg
{
   uint avutil_version()
}

And then a "default" implementation that is part of FFmpeg.AutoGen:

internal static class ffmpeg_dynamic_extern {
   [DllImport(AvutilDll]
	public static extern  uint avutil_version();
}

public class ffmpeg_dynamic: Iffmpeg
{
   uint avutil_version() =>  ffmpeg_dynamic_extern.avutil_version();
}

Those two classes one can copy to a properietary custom library and there link against static libs. (via AVutilDll="__Internal") Now the code that uses ffmpeg can work on an Iffmpeg interface (or a global static IFFmpeg instance somewhere) and works no matter whether you are in "free to use dynamic libraries" land or in "walled garden land".

Maybe there are other/better options. My basic requirement is just: I want to do platform independent (pure net6.0) library code that uses FFmpeg. In some cases this code will end up running on Windows or Linux, in some cases on iOS.

Is this a route that might be possible/interesting for FFmpeg.AutoGen? Or should I just do that stuff in my private code?

[Ruslan-B]

Right now FFmpeg.AutoGen already has OS X support so iOS presumably should work too. The only thing what needs to be done is a custom library loader and maybe ability to redefine GetFunctionDelegate but according to post it might work with a current one. The base for my guess is this post - https://stackoverflow.com/questions/27523308/using-dlsym-to-look-for-a-variable-in-a-statically-linked-library One thing though - you have to link everything with extra flag.

You can check MacNativeMethods and trace back how it is being used.

[digivod]

I checked MacNativeMethods. On iOS you can access a statically linked library like this:

[DllImport("__Internal", EntryPoint ="avutil_version")]
public static extern uint avutil_version();

and maybe you can use dlsym/dlopen magic to get a function delegate for the static library via the link you shared.

But the main problem remains: for this to work, the FFmpeg iOS static library must be statically linked against the FFmpeg.AutoGen library itself. Not against my library or my application. So the ios static library must become unchangeable part of the FFmpeg.AutoGen Nuget package itself. And you simply don't want that.

So instead the iOS static library must get into some propertiary assembly, outside of FFmpeg.AutoGen. But at the same time, I still want my library code like "read all data from an mp4 file with FFmpeg" to be platform independent and to have a ffmpeg "avformat_open_input" call that works the same everywhere. For that problem I'm looking for a solution (and proposed an idea)

I'll try to come up with a pull request or something similar to explain what I intend to do.

[Ruslan-B]

Okay, I think I got the gist. You need an abstraction for all exported functions. Let me experiment a just a bit as we do have delegates in place already and it seems to be feasible to do inverse decency injection by providing per platform nuget packages. I just wandering how sensitive iOS build chain as if static linked function does not present but DllImport will refer to it.

[digivod]

Exactly. I just tried it a bit myself. Worked like a charm. Here's a patch/idea on how to do it. IffmpegInterface.zip

[Ruslan-B]

I've created initial prototype: ffmpeg_abstractions branch. The idea behind is super simple: FFmpeg.AutoGen.Abstractions package is going to be an abstract API and will be common package. The rest - FFmpeg.AutoGen.Bindings.StaticallyLinked, FFmpeg.AutoGen.Bindings.DynamicallyLinked, FFmpeg.AutoGen.Bindings.DynamicallyLoaded Is going to be per bindings kind implementation.

Before to start usage you just need to call StaticallyLinkedBindings.Initialize(); or DynamicallyLoadedBindings.Initialize();, etc. It will do reverse dependency injection.

So you can use a pair of abstract API + bindings implementation per platform.

I coded it fast and dirty and implemented only StaticallyLinked and DynamicallyLoaded projects. So It still need work to remove spaghetti 🤣.

[digivod]

Had a quick look, and for me it's looking super! :-)

[Ruslan-B]

I uploaded FFmpeg.AutoGen.Abstractions and per binding kind FFmpeg.AutoGen.Bindings.StaticallyLinked, FFmpeg.AutoGen.Bindings.DynamicallyLinked, FFmpeg.AutoGen.Bindings.DynamicallyLoaded. So you have to pick one you needed per platform. Auxiliary API has a bit different naming's - i.e. fixed structures and function loading is different now.

[digivod]

Great! I'll have a look and report feedback. I also had crashes on mac for av_image_copy, most likely same issue as #182. Hopefully this is gone with the new rew ref based parameters.

[Ruslan-B]

Yes - now any fixed array is by ref if not const param.

[digivod]

done in ffmpeg_abstraction branch