AttributeError: TL1 , when building the project on MacOS , M1 device.

[DoggingDog]

I'm using :

• MacOS Ventura 13.2.1
• MacBook Air M1

When I execute the command : python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s

I got the message:

INFO:root:Compiling the code using CMake.
INFO:root:Downloading model HF1BitLLM/Llama3-8B-1.58-100B-tokens from HuggingFace to models/Llama3-8B-1.58-100B-tokens...
INFO:root:Converting HF model to GGUF format...
ERROR:root:Error occurred while running command: Command '['/opt/homebrew/anaconda3/envs/bitnet-cpp/bin/python3', 'utils/convert-hf-to-gguf-bitnet.py', 'models/Llama3-8B-1.58-100B-tokens', '--outtype', 'f32']' returned non-zero exit status 1., check details in logs/convert_to_f32_gguf.log

then I execute the command directly : python utils/convert-hf-to-gguf-bitnet.py models/Llama3-8B-1.58-100B-tokens --outtype f32

I got a AttributeError:

Traceback (most recent call last):
  File "/Users/doggingdog/workplace/py_workplace/BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1083, in <module>
    "tl1" : gguf.GGMLQuantizationType.TL1,
  File "/opt/homebrew/anaconda3/envs/bitnet-cpp/lib/python3.9/enum.py", line 429, in __getattr__
    raise AttributeError(name) from None
AttributeError: TL1

Did I miss something ? Has anybody try Bitnet on Mac and got the same problem ?

[potassiummmm]

It looks like gguf-py package is not installed correctly, please use the pip list command to check the gguf-py version, and check the contents of logs/install_gguf.log. To fix the problem, you can manually install gguf-py with python -m pip install 3rdparty/llama.cpp/gguf-py

[DoggingDog]

It looks like gguf-py package is not installed correctly, please use the pip list command to check the gguf-py version, and check the contents of logs/install_gguf.log. To fix the problem, you can manually install gguf-py with python -m pip install 3rdparty/llama.cpp/gguf-py

I tried it. guff-py successfully built:

Successfully built gguf
Installing collected packages: gguf
  Attempting uninstall: gguf
    Found existing installation: gguf 0.10.0
    Uninstalling gguf-0.10.0:
      Successfully uninstalled gguf-0.10.0
Successfully installed gguf-0.10.0

But the exception occur again:

Traceback (most recent call last):
  File "/Users/doggingdog/workplace/py_workplace/BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1083, in <module>
    "tl1" : gguf.GGMLQuantizationType.TL1,
  File "/opt/homebrew/anaconda3/envs/bitnet/lib/python3.9/enum.py", line 429, in __getattr__
    raise AttributeError(name) from None
AttributeError: TL1

is the guff version incompatible with project?

[srsng]

same problem.

[L4-1024]

same problem.

https://github.com/microsoft/BitNet/issues/25

this solved problem

[srsng]

run this command at a new dir of BitNet may help

git submodule update --init --recursive

it will help you clone and init the 3rdparty module (llama.cpp).

[DoggingDog]

run this command at a new dir of BitNet may help

git submodule update --init --recursive

it will help you clone and init the 3rdparty module (llama.cpp).

It works ! thx