BitNet

BitNet copied to clipboard

pip fail to install dependency

2

I need a few flags to successfully install dependencies.

yihuang

Converting HF model to GGUF doesn't seem to work?

6

I tried to follow the help instructions given by the setup_env.py script and ran the following command: ``` python setup_env.py --hf-repo microsoft/BitNet-b1.58-2B-4T -q i2_s INFO:root:Compiling the code using CMake. INFO:root:Downloading...

grctest

Using the latest version, I'm trying to setup BitNet following the setup_env.py help instructions: ``` python .\setup_env.py -help usage: setup_env.py [-h] [--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-7B-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit,tiiuae/Falcon3-10B-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-3B-1.58bit,tiiuae/Falcon3-1B-Instruct-1.58bit,microsoft/BitNet-b1.58-2B-4T}] [--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl2}] [--quant-embd]...

grctest

Can't run inference anymore

9

A few days ago i could run the inference as normal. Today is just doesn't work anymore. I downloaded the model and then setup according to the docs. LOGS: ```...

p-arndt

Build does not use correct clang

3

I downloaded and built the model (`microsoft/BitNet-b1.58-2B-4T-gguf`) using my Intel Mac following the instructions on README. I installed llvm using brew. ``` $ which clang /usr/local/opt/llvm/bin/clang $ clang --version Homebrew...

kmsngh

I used microsoft/bitnet-b1.58-2B-4T-gguf model Setup the environment： `python setup_env.py -md ./models/BitNet-b1.58-2B-4T/ -q i2_s` usage with: `python run_inference.py -m ./models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv -n 1024` and output...

wangpeng813

fix for Microsoft's incompetence

2

``` export CC=clang export CXX=clang++ cmake -B build -DGGML_BITNET_ARM_TL1=ON -DCMAKE_C_COMPILER=clang-19 -DCMAKE_CXX_COMPILER=clang++-19 CC=/usr/bin/clang CXX=/usr/bin/clang++ python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s ```

gilbrotheraway

why two models here are same

1


RQiand

Unable to build the project

2

Hi I am unable to build the project (bitnet-cpp) PS C:\Users\anuthakur\Documents\msbitnet\BitNet> python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s Traceback (most recent call last): File "C:\Users\anuthakur\Documents\msbitnet\BitNet\setup_env.py", line 232, in main() File "C:\Users\anuthakur\Documents\msbitnet\BitNet\setup_env.py",...

anunay1

Format conversion issue after downstream SFT

9

I used my own dataset to finetune the model `bitnet-b1.58-2B-4T-bf16` for downstream task. The saved checkpoint directory is as follows: ``` shell path/to/my/ckpt ├── chat_template.jinja ├── config.json ├── generation_config.json ├──...

LiuZhihhxx