server: Windows 7 compatibility

[Zor-X-L]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.

I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.

In theory it can run on XP or Vista, but I don't have test environments.

• [x] I have read the contributing guidelines
• Self-reported review complexity:
  • [ ] Low
  • [x] Medium
  • [ ] High

[ngxson]

I'm not very confident with this change, because httplib is copied from external repo: https://github.com/yhirose/cpp-httplib

All the changes made here in llama.cpp will be overwritten when we sync with upstream source code

[Zor-X-L]

@ngxson the upstream repo has fixed the issue (yhirose/cpp-httplib#1869), set -D_WIN32_WINNT=_WIN32_WINNT_WIN7 -DWINVER=_WIN32_WINNT_WIN7 will make it call the proper API on Win7.

After I update the httplib to the latest version, I find GGML_WIN_VER was set to 0x602 which is Win8 because PrefetchVirtualMemory. But the call to PrefetchVirtualMemory is dynamic so actually it doesn't need -D_WIN32_WINNT=_WIN32_WINNT_WIN7, but to define WIN32_MEMORY_RANGE_ENTRY and PWIN32_MEMORY_RANGE_ENTRY when _WIN32_WINNT < _WIN32_WINNT_WIN8.

I set GGML_WIN_VER to _WIN32_WINNT_WIN7 (0x601 doesn't work, dunno why) and add _WIN32_WINNT=${GGML_WIN_VER} WINVER=${GGML_WIN_VER} to places that need it. Makefile also edited.

I compile the project on win11 with vs2022 and msys2-mingw64 and both executables works on win7 (msys2-ucrt doesn't work).

What do you think?

[sharpden]

2567073 works on my Windows 7, thank you!

[mhnoni]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile.

I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine.

In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

[sharpden]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile. I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine. In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

Last CUDA that officially supports Windows 7 is 10.2, but build with 10.2 is probably broken long ago https://github.com/ggerganov/llama.cpp/issues/4123 (For XP x32 - 6.5, x64 - 6.0)

[mhnoni]

Last CUDA that officially supports Windows 7 is 10.2, but build with 10.2 is probably broken long ago #4123 (For XP x32 - 6.5, x64 - 6.0)

oh, that makes sense, thanks, I spent my whole day trying to find the issue and I couldn't I though I have some issue on my side, IO was getting "nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified" and I thought I have the wrong cuda, I even tried to compile it using 11.2 on windows 10 but couldn't either (the old version of llama.cpp that supported win 7). I guess its not easy anymore to let llama.cpp to support win 7. I wonder if there is any other runtimes that I could use with win 7 that support cuda.

[zhaofeng-shu33]

I want to run the server on a Win7 box, and the API calls to CreateFile2/CreateFileMappingFromApp/MapViewOfFileFromApp in httplib.h prevent this (need Win8 at least), so I change them to CreateFileW/CreateFileMappingW/MapViewOfFile. I tested the code on Win7 with qwen2-0_5b-instruct-q4_k_m (about 400MiB) and on Win11 with Yi-1.5-34B-Chat-IQ4_XS (about 18GiB), they all work fine. In theory it can run on XP or Vista, but I don't have test environments.

Hi Zor-X-L

Any chance you remember which cuda version you used to compile it? or you only test it on CPU? I was able to build the old version with the help of your pr for CPU but not cuda.

And do you think you can mak a pr for the latest version too for win 7?

Thanks!

The latest version has merge conflict with this pr, but I think these conflicts are not difficult to solve. As others methoned, it is difficult to support win7+cuda build version, but win7+pure cpu can be reserved.I Based on this pr, I suggest add some "CMake switch" to differentiate cuda build and non-cuda windows build. By doing so, all CICD can pass.

[sharpden]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

[mhnoni]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

Could you please link the pr? does your pr let us compile it and run on windows 7 including the webclient? And which cuda version we will need for it? my problem last time trying to compile for Win 7 was with cuda.

[zhaofeng-shu33]

I have successfully built quite recent b4568 with patches only for 4 CMakeFiles.txt and 1 Makefile. @Zor-X-L , @ngxson , could you please merge this PR already? 8 months should be enough for 9 lines change.

Could you please link the pr? does your pr let us compile it and run on windows 7 including the webclient? And which cuda version we will need for it? my problem last time trying to compile for Win 7 was with cuda.

Please remove the following lines in CMakeLists.txt

if(MSVC)
    add_compile_options("$<$<C_COMPILER_ID:MSVC>:/utf-8>")
    add_compile_options("$<$<CXX_COMPILER_ID:MSVC>:/utf-8>")
endif()

nvcc from cuda 10.2 cannot recognize /utf8 option.

I have successfully compile this PR version using VS2017 + Cuda 10.2 on win7. Some other macros also neeed to be patched.