matjazbo comments

Repositories
Issues
Comments

Results 2 comments of


                                            matjazbo

Using CUDA, but GPU shows near 0% usage

I also have this issue, GPU memory is allocated, but only CPU is used for inference. [ollama.log](https://github.com/ollama/ollama/files/14108568/ollama.log)

Using CUDA, but GPU shows near 0% usage

@easp you might be correct, although when running Phi-2, I didn't see any GPU usage, neither in task manager nor in nvidia-smi. I'm using 4070 with 12GB which seem to...