Rover van der Noort

I think this is a great idea! For my own project I was looking for something like this, but then instead of cloning the option, I was looking to count...

> Makefile needs to be modified because 10's nvcc doesn't have the --forward-unknown-to-host-compiler option, nor the -arch=native [cuda10.patch](https://github.com/ggerganov/llama.cpp/files/13400052/cuda10.patch) Hi I tried to use your patch to compile on my Nvidia...

It looks like a promising patch, thanks! I can only test this in the new year, unfortunately, but I'll let know the results by then.

UPDATE: Managed to compile now! Needed to export the gcc installation for make by: ``` export CC=/usr/local/bin/gcc export CXX=/usr/local/bin/g++ ``` I've installed gcc 8.5 from source ``` gcc -v Using...

According to the nvml docs it is indeed mJ instead of uJ  Maybe a good idea to add to the docs since I had to search in pyRAPL and...

I'm using this board: https://developer.nvidia.com/embedded/jetson-nano-developer-kit I would very much appreciate the help!

Potentially related?: https://github.com/ggerganov/llama.cpp/issues/4123 EDIT: I've run it with the patch file from this issue, but get some new errors, which I don't know how to solve. Did anyone else make...

@staviq any luck so far on your end?

Thanks for all the suggestions, and to conclude how to install on an Nvidia Jetson Nano with ubuntu 18, cuda10.2 1. Compile and install gcc 8.5 from source and export...

> @rvandernoort What kind of performance do you get out of this? > > In my case running it without GPU offloading is actually faster than with it. Do you...