BitNet
BitNet copied to clipboard
microsoft
Official inference framework for 1-bit LLMs
The CPU with ARM architecture failed to run `env_setup.py`. Make the following modifications: 1. Change the code from ``` posix_memalign(&ptr, 64, size); return ptr; ``` to ``` if posix_memalign(&ptr, 64,...
As title, I can not build the project on Raspberry Pi 4 with Raspberry Pi OS (Debian 12 (bookworm)) and gcc 12. command: `python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s` ```...
cc: error: unrecognized command-line option ‘-Wunreachable-code-break’; did you mean ‘-Wunreachable-code’? cc: error: unrecognized command-line option ‘-Wunreachable-code-return’; did you mean ‘-Wunreachable-code’? gmake[2]: *** [3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/build.make:76: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:759: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/all]...
When I prompt it to generate more output, like a python script, it abruptly truncates the responses, returning to a prompt. I can then press `Ctl+D` or `Enter` and the...
python setup_env.py --hf-repo 1bitLLM/bitnet_b1_58-3B --quant-type i2_s INFO:root:Compiling the code using CMake. ERROR:root:Error occurred while running command: Command '['cmake', '--build', 'build', '--config', 'Release']' returned non-zero exit status 2., check details in...
After the step `python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s1` I see by `htop` and `Activity Monitor` that the CPU cores are barely busy. How can I start a build to...
Hi everyone I got a mac M1 and I tried to build bitnet, everythings ran fine until I ran `python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s` When I run it, I...
This step been running for 24 hours: `python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s1` How long should this take? How do I know if it's making progress?
I setup the Conda environment and even reinstalled the model but all I get is "G" as an output? 
I tried to deploy BitNet on my arm64 laptop. But suddenly I run into issues: I followed the guidence on the webpage and typed the instruction one by one correctly....
