pip install llama-cpp-python got stuck forever at "Configuring CMake" in docker

[jiafatom]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ X] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [X ] I carefully followed the README.md.
• [ X] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [ X] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Please provide a detailed written description of what you were trying to do, and what you expected llama-cpp-python to do.

Current Behavior

Please provide a detailed written description of what llama-cpp-python did, instead.

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

• Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 96 On-line CPU(s) list: 0-95 Vendor ID: AuthenticAMD Model name: AMD EPYC 7V12 64-Core Processor CPU family: 23 Model: 49 Thread(s) per core: 1 Core(s) per socket: 48 Socket(s): 2 Stepping: 0 BogoMIPS: 4890.89 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonst op_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalig nsse 3dnowprefetch osvw topoext perfctr_core ssbd vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru arat umip rdpid Virtualization features:
Hypervisor vendor: Microsoft Virtualization type: full Caches (sum of all):
L1d: 3 MiB (96 instances) L1i: 3 MiB (96 instances) L2: 48 MiB (96 instances) L3: 384 MiB (24 instances) NUMA:
NUMA node(s): 4 NUMA node0 CPU(s): 0-23 NUMA node1 CPU(s): 24-47 NUMA node2 CPU(s): 48-71 NUMA node3 CPU(s): 72-95 Vulnerabilities:
Gather data sampling: Not affected Itlb multihit: Not affected L1tf: Not affected Mds: Not affected Meltdown: Not affected Mmio stale data: Not affected Reg file data sampling: Not affected Retbleed: Mitigation; untrained return thunk; SMT disabled Spec rstack overflow: Vulnerable: Safe RET, no microcode Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Spectre v2: Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected Srbds: Not affected Tsx async abort: Not affected

• Operating System, e.g. for Linux:

$ uname -a Linux 8cad064ec6fd 6.6.57.1-7.azl3 #1 SMP PREEMPT_DYNAMIC Thu Jan 2 17:46:33 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

• SDK version, e.g. for Linux:

$ python3 --version
Python 3.11.10
$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
$ g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.

Failure Information (for bugs)

Run CMAKE_ARGS="-DGGML_CUDA=on -DLLAVA_BUILD=OFF -DCMAKE_CUDA_ARCHITECTURES=\"70;75;80\"" pip install llama-cpp-python --verbose and it stuck forever at: Running command Building wheel for llama-cpp-python (pyproject.toml) *** scikit-build-core 0.10.7 using CMake 3.30.5 (wheel) *** Configuring CMake...

Steps to Reproduce

In docker environtment, Run CMAKE_ARGS="-DGGML_CUDA=on -DLLAVA_BUILD=OFF -DCMAKE_CUDA_ARCHITECTURES=\"70;75;80\"" pip install llama-cpp-python --verbose

Note: Many issues seem to be regarding functional or performance issues / differences with llama.cpp. In these cases we need to confirm that you're comparing against the version of llama.cpp that was built with your python package, and which parameters you're passing to the context.

Try the following:

1. git clone https://github.com/abetlen/llama-cpp-python
2. cd llama-cpp-python
3. rm -rf _skbuild/ # delete any old builds
4. python -m pip install .

It stuck here also.

Failure Logs

Please include any relevant log snippets or files. If it works under one configuration but not under another, please provide logs for both configurations and their corresponding outputs so it is easy to see where behavior changes.

Also, please try to avoid using screenshots if at all possible. Instead, copy/paste the console output and use Github's markdown to cleanly format your logs for easy readability.

Example environment info:

llama-cpp-python$ git log | head -1
commit 0580cf273debf4a7f2efcdfd5ef092ff5cedf9b0

llama-cpp-python$ python3 --version
Python 3.11.10

llama-cpp-python$ pip list | egrep "uvicorn|fastapi|sse-starlette|numpy"
numpy                                  1.23.5
opentelemetry-instrumentation-fastapi  0.50b0

fastapi                                0.115.7
numpy                                  1.23.5
opentelemetry-instrumentation-fastapi  0.50b0
sse-starlette                          2.2.1
uvicorn                                0.34.0

llama-cpp-python/vendor/llama.cpp$ git log | head -3
commit 0580cf273debf4a7f2efcdfd5ef092ff5cedf9b0
Author: Andrei Betlen <[email protected]>
Date:   Wed Jan 8 16:53:54 2025 -0500

[Rocketknight1]

I'm also getting this issue, and am not familiar enough with cmake to diagnose it!

[kk3dmax]

I have the same issue with v0.3.7.0 even with pip install.

[MisterDr]

Same also, it even chews 256GB of ram while trying to install.

[MisterDr]

I have able to proceed with install by updating ninja to the 1.12.2 version, and adding this in the CMakeLists.txt:

find_package(OpenMP REQUIRED) if(OpenMP_CXX_FOUND) message(STATUS "Found OpenMP: ${OpenMP_CXX_FLAGS}") # Append the OpenMP flags to the global C++ compile flags set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}") # Append the OpenMP flags to the global linker flags set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_CXX_FLAGS}") endif()