Failing to build llama-stack on Mac OS X

[genbit]

Followed Getting Started guide, tried to build locally on Mac OS 15.0 (M1):

  Using cached fairscale-0.4.13.tar.gz (266 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
ERROR: Could not find a version that satisfies the requirement fbgemm-gpu==0.8.0 (from versions: none)
ERROR: No matching distribution found for fbgemm-gpu==0.8.0
Failed to build target my-local-stack with return code 1

[linsssbei]

+1. not sure if fbgemm-gpu is on macos

[californias66]

I'm getting the same issue, no way macOS can't build this? What am I doing wrong?

[bhargavyagnik]

The current inference supports only Cuda-based drivers, if you want to build on macos, you can use support with external inference like Ollama. It would work with that.

[genbit]

Thanks @bhargavyagnik for pointers.

This is how it worked for me:

# shows me list of templates available for build command
$ llama stack build --list-templates

# use local-ollama inferencing
$ llama stack build --template local-ollama --name my-local-stack

This builds a stack using meta-reference for everything except using remote::ollama for inferencing.

[californias66]

Thank you fam you're all GOATS fsho 🙏🏽🔥😮‍💨