How to handle `CUDA error: out of memory`?

[yukozh]

Description

CUDA error: out of memory, the whole process terminated.

Is there a way to catch this error instead of terminate the process?

[martindevans]

LLamaSharp catches llama.cpp error codes where possible, and turns them into C# exceptions that you can handle in the normal way. However, some errors in C++ simply abort the process with no possibility for C# to catch it. This looks like one of those cases.

I would suggest adding a comment this this issue with as much detail about the error as possible. In particular a stack trace, if you can get one, otherwise at least trace through the C# code to where it calls a native C++ method.

[github-actions[bot]]

This issue has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.