Edward Sarker

Same issue. My setup was with docker -> portainer -> pihole and it just never told me why I don't get a bind. Thanks, this should be easier to find,...

My 2 cents, seems to work on the clusters created in the NS by etcd-operator, doesn't seem to work on my external clusters, I gave my backupCR the endpoints, the...

Hello! This is an amazing PR. But I'm having trouble understanding the steps for this ``` By default, this change supports 1 concurrent requests to a loaded model, which can...

Docker server logs; ``` ❯ docker logs -f ollama time=2024-04-19T23:00:21.844Z level=INFO source=images.go:817 msg="total blobs: 162" time=2024-04-19T23:00:21.845Z level=INFO source=images.go:824 msg="total unused blobs removed: 0" time=2024-04-19T23:00:21.846Z level=INFO source=routes.go:1143 msg="Listening on [::]:11434 (version...

And to be clear, my goal is to be able to load a 38GB model split across 2 GPUs. So essentially I want scenario A) most layers in GPU 1...

I mean that would still be useful. Could I in theory do for example the new llama3:70b model if I wanted to run it in any of these following scenarios?...

Amazing. I'll close this. Though I know I initially wanted CUDA+RoCM to play nice, we're probably really far from that at the moment and its not a specific to ollama...

Seeing the same. Docker is my preferred deployment method as it covers ROCm and CUDA in a single image without much dependencies, but as soon as I upgraded my image,...

Is there possibly a way to trigger the docker build? I'm running into issues trying to build it locally. Docker build is still 8 days old.

> All my problems seem to disappear when I limit the amount of memory for ollama. My 7900 XT has 20GB, so I have limited it now to 18GB like...