appstore icon indicating copy to clipboard operation
appstore copied to clipboard

Create quadlet configuration for Ollama

Open p5 opened this issue 1 year ago • 1 comments

I wanted to provide the Ollama quadlet definitions because it shows how we can attach GPUs to the quadlets, plus changing the selinux configuration and using the .d directory for switching between Nvidia and ROCm.

p5 avatar May 06 '24 18:05 p5

@p5 these are great, I think the ollama.container.d/ only effects the parts outside of the quadlet, IE Things like [Service] and other fields.

The quadlet is changing the IMAGE=foobar line into a podman command to execute foobar. Systemd will read the ollama.container.d/ when the service is starting.

rhatdan avatar May 15 '24 21:05 rhatdan

@p5 whats up with this one?

rhatdan avatar Oct 16 '24 20:10 rhatdan

Will close this as I don't know when I'll get round to finishing it (and I completely forgot I opened it in the first place :facepalm:). Apologies

p5 avatar Oct 16 '24 20:10 p5

You could always take a look at RamaLama... :^)

rhatdan avatar Oct 16 '24 20:10 rhatdan

Starting some investigation into getting CUDA running on ramalama, and will certainly look into contributing a quadlet if it all works :)

p5 avatar Oct 16 '24 22:10 p5