BitNet

BitNet copied to clipboard

[DeepSeek R1 1.58bit](https://huggingface.co/unsloth/DeepSeek-R1-GGUF) Note: Unsloth uses some sort of dynamic quantization

vegax87

Hi BitNet team, thanks for the great job! I know you've done a lot on your [fork of llama.cpp](https://github.com/Eddie-Wang1120/llama.cpp) and it's difficult to merge back to upstream due to conflicts,...

southwolf

ByteDance flux 1.58 model?

1

Does it work with this code? https://github.com/Chenglin-Yang/1.58bit.flux

hrstoyanov

help cmak

1

i try many way but i have a problem wen i use (bitnet-venv) (ai) root@mgtm3na:~/BitNet# python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s and this is the result INFO:root:Compiling the code using...

ahmedmetwali1

# Environment * CPU: NVIDA Grace CPU(72thread) * Model: Llama3-8B-1.58-100B-tokens with TL1 * Prompt: AI is going to * N_Predict: 128 * Threads: 1, 2, 4, 8, 16, 32, 64,...

y-vectorfield

I'm not sure whats wrong i have all of the stuff i need from VS22        ```CMake Error at CMakeLists.txt:2 (project): Generator NMake Makefiles...

RealMrCactus

I wanted to quickly test the recently released model on a Linux machine via the terminal, so I put together a simple [Colab notebook](https://colab.research.google.com/drive/1YWEISZD3OTr4L11g8-PK4l0H7dQXarUo#scrollTo=BWF1MMYsbDaf) that includes the necessary installation and...

neonwatty

Unable to run setup_env.py

12

I have followed the instruction from https://github.com/microsoft/BitNet?tab=readme-ov-file#build-from-source but stuck at step to build the project shell log: ``` (bitnet-cpp) user@bitnet1:~/BitNet$ python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s INFO:root:Compiling the code using...

Kahdeg-15520487

pthread_attr_setaffinity_np: symbol not found

1

 Traceback (most recent call last): When I tried python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s, Got the above problem. Could you please give some advice. File "/home/workspace/BitNet/utils/convert-hf-to-gguf-bitnet.py", line 20, in...

zhaoxuejun1234

arm device compile error.

11

BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:190:43: error: cannot convert ‘int16x8_t’ to ‘const int8x16_t’ in initialization 190 | const int8x16_t vec_zero = vdupq_n_s16(0x0000); | ~~~~~~~~~~~^~~~~~~~ | | | int16x8_t BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:232:22: error: invalid operands to binary +...

theone10zero