Compile bug: ggml-cuda requires the language dialect "CUDA17"

[hh2712]

Git commit

$git rev-parse HEAD 855cd0734aca26c86cc23d94aefd34f934464ac9

Operating systems

Linux

GGML backends

CUDA

Problem description & steps to reproduce

I'm trying to install llama.cpp with cuda configuration but getting error with cmake compilation requiring language dialect "CUDA17". I have updated my cuda to the latest version cuda-12.8, btw.

First Bad Commit

No response

Compile command

cmake -B build -DGGML_CUDA=ON

Relevant log output

-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native
-- CUDA Toolkit found
-- Using CUDA architectures: 52;61;70;75
-- CUDA host compiler is GNU 8.4.0

-- Including CUDA backend

-- Configuring done (0.4s)
CMake Error in ggml/src/ggml-cuda/CMakeLists.txt:
  Target "ggml-cuda" requires the language dialect "CUDA17" (with compiler
  extensions).  But the current compiler "NVIDIA" does not support this, or
  CMake does not know the flags to enable it.


-- Generating done (0.7s)
CMake Generate step failed.  Build files cannot be regenerated correctly.

[highfillgoods]

Had the same error as you, hitting a brick wall while installing llama.cpp.

Steps did to fix the error ggml-cuda requires the language dialect "CUDA17"

While using Ubuntu 20.04 .....5.15.0-1074-oracle. Installed older cuda + display driver version 535.

1. uninstall nvidia completely & uninstall cuda. Restart.

2. downlaod the 4GB cuda + driver file [](wget https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux.run )

3. kill your window manager.

sudo service lightdm stop

4. Install cuda driver, and display driver 535. Reboot
5. Point /usr/local to the right cuda folder which is now the new cuda-12.2. Since the old one was still linked in.

echo 'export PATH=/usr/local/cuda-12.2/bin:$PATH' >> ~/.bashrc
exec bash

6. Update cmake 3.16 to cmake version 3.27.5
7. Make sure your nvcc --version + nvida-smi cuda version are the same. For me its 12.2

$nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Jun_13_19:16:58_PDT_2023
Cuda compilation tools, release 12.2, V12.2.91
Build cuda_12.2.r12.2/compiler.32965470_0

$nvidia-smi
Mon Feb 10 16:49:37 2025       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.216.01             Driver Version: 535.216.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+

The output doing the cmake

 cmake .. -DGGML_CUDA=ON \
>   -DBUILD_SHARED_LIBS=OFF \
>   -DLLAMA_CURL=ON \
>   -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.2

-- The C compiler identification is GNU 10.5.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.25.1") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Found CUDAToolkit: /usr/local/cuda-12.2/include (found version "12.2.91") 
-- CUDA Toolkit found
-- Using CUDA architectures: native
-- The CUDA compiler identification is NVIDIA 12.2.91
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-12.2/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CUDA host compiler is GNU 10.5.0

-- Including CUDA backend
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.68.0")  
-- Configuring done (4.5s)
-- Generating done (0.2s)

[hh2712]

THX, made it work by reinstall cuda and manually assign couple of cuda variable path in the CMakeList.txt as well. (Did not uninstall older cuda version for other colleague in my case.)