LLamaSharp Embeding LLamaSharp inside .NET MAUI Android App

Description

I am a .NET MAUI develpoer and I am interested in embeding a LLM inside my application with LLamaSharp. After build llama.cpp with ndk , How can I embed libraries on my application??

Oct 17 '24 11:10 mmxkarimi

Description

I am a .NET MAUI develpoer and I am interested in embeding a LLM inside my application with LLamaSharp. After build llama.cpp with ndk , How can I embed libraries on my application??

I'm also interested, did you manage to use it?

Oct 28 '24 13:10 andersonvieiragomeslopes

After solving some problems with embeding model inside app , I will publish it into my github.

Oct 30 '24 19:10 mmxkarimi

Looking forward to your solution to this problem. I am not sure if llamasharp would work out of the box on android/ios, but maybe it is possible to compile llama.cpp for the specified platform?

At least llama.cpp runs under Termux on Android...

I will keep an eye on your repository: https://github.com/Karimiprogramer/Sigma

Nov 17 '24 23:11 Darlanio

Tested on 0.19 without compiling the binaries myself and got the exception System.PlatformNotSupportedException: 'Operation is not supported on this platform.' on the ModelParams constructor.

Nov 22 '24 06:11 Darlanio

Looking foward to Android supported nuget libraries.

Nov 29 '24 15:11 Darlanio

Is there a solution for iOS?

Jan 30 '25 21:01 homerokzam

"Hey @AmSmart, could you join this discussion here? I saw that you had success with this Maui Android Backend Support."

Feb 01 '25 16:02 andersonvieiragomeslopes

Yeah, I have a PR that brings Android support for this library. Just that I haven't had enough time to write the tests and all. I'll open my PR this weekend and see if it's good enough to get merged. Also planning an IOS update in the future.

Feb 01 '25 17:02 AmSmart

Yeah, I have a PR that brings Android support for this library. Just that I haven't had enough time to write the tests and all. I'll open my PR this weekend and see if it's good enough to get merged. Also planning an IOS update in the future.

Any updates on this, @AmSmart ?

This will be a really cool feature!

Feb 22 '25 11:02 amirvenus

I have successfully built CPU-based versions of libllama and libgguf for Android and imported them into my .NET MAUI application. However, I am facing some challenges in utilizing these libraries effectively within my app.

I would greatly appreciate any guidance or support you can provide to help me overcome these issues. Thank you!

Mar 16 '25 17:03 mmxkarimi

One thing to check - did you build exactly the llama.cpp version listed in the table at the bottom of the readme? Each version of LLamaSharp is only compatible with one specific version of llama.cpp!

Mar 18 '25 14:03 martindevans

I wish I had time to check this now, but will have to wait until I have time in April.

Mar 18 '25 16:03 Darlanio

Update: The extra demand here gave me the motivation to complete my PR #1130 introducing Android support. Will still require a bit of back and forth but I hope it can get merged soon enough.

I've built this nupkgs off my PR so anyone blocked can use them to experiment temporarily. Reiterating that these are not official packages from the repo so it is definitely worth waiting for the next release with official Android support.

Mar 20 '25 10:03 AmSmart

This issue has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.

May 23 '25 00:05 github-actions[bot]