[WinError 193] When trying to run the high level API example with vicuna

[ChilliSawse]

I ran pip install llama-cpp-python and the installation was a success, then I created a python file and copied over the example text in the readme. The only change I made was the model path to the vicuna model I am using and when I try to run the script I end up getting this error:

  File "C:\Users\Chula\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\llama_cpp\llama_cpp.py", line 36, in _load_shared_library
    return ctypes.CDLL(str(_lib_path))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Chula\Desktop\code_projects\Vicuna_1\test.py", line 1, in <module>
    from llama_cpp import Llama
  File "C:\Users\Chula\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\llama_cpp\__init__.py", line 1, in <module>
    from .llama_cpp import *
  File "C:\Users\Chula\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\llama_cpp\llama_cpp.py", line 46, in <module>
    _lib = _load_shared_library(_lib_base_name)
  File "C:\Users\Chula\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\llama_cpp\llama_cpp.py", line 38, in _load_shared_library
    raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'C:\Users\Chula\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\llama_cpp\llama.dll': [WinError 193] %1 is not a valid Win32 application
Press any key to continue . . .

I'm fairly new to all this so may have done something wrong but can seem to find a fix anywhere.

[abetlen]

That's very strange, it looks like the error is occuring here https://github.com/abetlen/llama-cpp-python/blob/main/llama_cpp/llama_cpp.py#L36 you could try checking out the python ctypes package https://docs.python.org/3/library/ctypes.html and see if you can use something like

I've looked around and the only thing that I've found is that the dll may have been built incorrectly (?) can you try to pip install --upgrade --force-reinstall --no-cache-dir llama-cpp-python

[ChilliSawse]

Hey, sorry for the delay! I've given the pip install --upgrade --force-reinstall --no-cache-dir llama-cpp-python a shot and it installs without any errors in Visual Studio but I am still getting the same error when I try to run the code...

[gjmulder]

@ChilliSawse can you:

1. Confirm the properties of the file llama_cpp\llama.dll are in fact of a WIndows 32 bit DLL
2. If so, try and reproduce the problem again with the latest code.

[gjmulder]

Closing. Reopen if the problem is still occuring with the latest code.

[jamesqh]

I am having the same problem with the latest code.

My CPU has no AVX2, but I'm able to build and run llama.cpp with the right cmake args (build log) - so the build environment should be working?

But when I install llama-cpp-python in the same environment and try to use it, I get the "not a valid Win32 application" error. I get it after a simple install:

(venv) D:\llamastuff\test>pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir
Collecting llama-cpp-python
  Downloading llama_cpp_python-0.1.54.tar.gz (1.4 MB)
     ---------------------------------------- 1.4/1.4 MB 1.5 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0 (from llama-cpp-python)
  Downloading typing_extensions-4.6.1-py3-none-any.whl (31 kB)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.54-cp310-cp310-win32.whl size=218578 sha256=2e17f25276010581b0602e727d6610e61d08a440f3d326dc1a21e144b81247c0
  Stored in directory: C:\Users\Vardogger\AppData\Local\Temp\pip-ephem-wheel-cache-y7l16kv2\wheels\21\d2\0b\120c33e6d5328d4443ea58703e4cd26b2d5f33b98e112d95f2
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, llama-cpp-python
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.6.1
    Uninstalling typing_extensions-4.6.1:
      Successfully uninstalled typing_extensions-4.6.1
Successfully installed llama-cpp-python-0.1.54 typing-extensions-4.6.1

(venv) D:\llamastuff\test>python test.py
Traceback (most recent call last):
  File "D:\llamastuff\test\venv\lib\site-packages\llama_cpp\llama_cpp.py", line 60, in _load_shared_library
    return ctypes.CDLL(str(_lib_path), **cdll_args)
  File "D:\python\python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\llamastuff\test\test.py", line 1, in <module>
    from llama_cpp import Llama
  File "D:\llamastuff\test\venv\lib\site-packages\llama_cpp\__init__.py", line 1, in <module>
    from .llama_cpp import *
  File "D:\llamastuff\test\venv\lib\site-packages\llama_cpp\llama_cpp.py", line 73, in <module>
    _lib = _load_shared_library(_lib_base_name)
  File "D:\llamastuff\test\venv\lib\site-packages\llama_cpp\llama_cpp.py", line 62, in _load_shared_library
    raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'D:\llamastuff\test\venv\Lib\site-packages\llama_cpp\llama.dll': [WinError 193] %1 is not a valid Win32 application

And I get the exact same result installing with FORCE_CMAKE=1 and CMAKE_ARGS="-DLLAMA_AVX2=OFF", so it doesn't seem like it's an AVX thing. I've managed to install llama-cpp-python in the past so this seems like a new issue.

Not sure exactly what properties you would like to be confirmed, but I threw the offending llama.dll into pestudio and it doesn't look obviously broken?

Here is the dll anyway

[jamesqh]

Figured this out myself. The error message was kinda backwards: the problem isn't that the dll isn't a valid Win32 application, but that it is and it shouldn't be!

I was using "Developer Command Prompt for VS 2022", and I guess that compiles it to 32 bit. The solution is to use x64 Native Tools Command Prompt for VS 2022 instead. pip install llama-cpp-python in that environment installs a dll that fails to load models because of "Windows Error 0xc000001d", which in my case seems to mean that the binary is trying to use AVX2 without success. set FORCE_CMAKE=1 and set CMAKE_ARGS="-DLLAMA_AVX2=OFF" followed by the pip install gives me a llama-cpp-python that... well, fails to load a model due to "Windows Error 0xe06d7363", but that seems like a separate issue and at least it's actually importing the module now.

(Windows Error 0xe06d7363 was due to all the models I was trying to use being wrong versions. My kingdom for a set of standardised tiny test models)

[SpaceCowboy850]

I've done this:

set FORCE_CMAKE=1 set CMAKE_ARGS="-DLLAMA_AVX2=OFF -DLLAMA_CUBLAS=ON" pip install --upgrade --force-reinstall --no-cache-dir llama-cpp-python

The CMAKE_ARGS, I've tried empty, with only -DLLAMA_CUBLAS=ON (which causes a cudafe++ error), with only -DLLAMA_AVX2=OFF.

I've tried with a VS2022 Command Prompt, a VS2019 Command Prompt, and a x64_x86_Cross Tools Commapn Prompt for VS2022.

In every scenario I can't get a clean load of "from llama_cpp import Llama" without getting the %1 is not a valid Win32 application.

Could it be that CMAKE is defaulting to building the Win32 version, and we need to get it to build the x64 platform? I'm unclear what to pass to make that happen.

I did get all of this working in Ubuntu 23.10, so that's something. It would just be nice to get it running on my Windows box as well.

[julian-passebecq]

did you find the solution @SpaceCowboy850 ? i have the same issue

[SpaceCowboy850]

No...and I was just trying to get latest again, came across the same bug, again, and found this surprisingly familiar page only to see I had complained about the same thing 4 months ago. :(

I ended up at the time just using WSL, which Llama_cpp_python installs and runs fine on, but it would be way more convenient if I could run this via native windows instead.

EDIT: Prebuilt wheel works for me: pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu123

See: https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#supported-backends Expand "CUDA" and follow along the prebuilt instructions.