m0nsky
m0nsky
Running into the same issue. Training w/ Unsloth (LLaMA-Factory) through WSL succesfully spills over to system RAM with CUDA `Sysmem Fallback Policy` enabled, allowing me to train a 16k context...
> @m0nsky OOO interesting so `non_blocking = False` works?? Hmm maybe I should make a new method called `"unsloth-wsl"` for WSL people, to use blocking calls. You will get some...
Using this PR, I was able to switch between AVX2 and CUDA in an Unity game. 👍
My tests were done on Windows 10. Unfortunately I don't have any other platforms to test on.
@AsakusaRinne I think there is an even better use case for this now, since Vulkan support was introduced to LLamaSharp in 0.14 (July 16). Is there any chance you could...
Maybe it would be better to skip the Vulkan device check for now (but leave the code in place), and leave it to the user to enable Vulkan on the...
@emulk @CastIehard this should now be fixed in the LLamaSharp 0.15 release from 3 days ago.
Hi @LucaMaccarini , does this PR need any additional changes? I'm not sure what you mean by that last note (have you found out if we need to start with...
Yes, absolutely, here are my steps: - Download `nuget.exe` and add to PATH (environment variable, Windows 11) - Clone your WIP branch, build the project - Pack the nuget packages,...
> Thank you both @m0nsky and @LSXAxeller for your feedback. I have fixed the problem, which was caused by the .props files for the Vulkan and CUDA12 packages having incorrect...