BitNet
BitNet copied to clipboard
microsoft
Official inference framework for 1-bit LLMs
Change the default number of tokens to predict. -1 is the default in llama.cpp, any other value will confuse people. Closes #264.
Hello, I am attempting to run the BitNet model on a Windows 11 machine with an Intel N6000 CPU, which does not have AVX/AVX2 support. The installation completes, but inference...
Hi, When I am using cnv mode and it is correctly giving the exact answer. But when I am using llama server mode and triggering endpoint with question, It returns...
Description: Tried the fwk built with I2_S kernel. Below are a few cases for which got strange responses. Would you please help me understand how an I use the framework...
Description: Build fails when using TL2 Steps to reproduce: python setup_env.py -md models/BitNet-b1.58-2B-4T -q tl2 Error: NFO:hf-to-gguf:Loading model: BitNet-b1.58-2B-4T Traceback (most recent call last): File "utils/[convert-hf-to-gguf-bitnet.py](https://convert-hf-to-gguf-bitnet.py/)", line 1168, in main()...
Hi, and thanks for the great work on BitNet! I'm trying to fine-tune `microsoft/bitnet-b1.58-2B-4T-bf16` using a Korean dataset (`nlpai-lab/kullm-v2`) with SFTTrainer. However, during training, the loss remains around **3.3 to...
This commit introduces three new files: 1. README_ko.md: A Korean translation of the original README.md. 2. Pretrain-Tuning.md: A document in Korean outlining methods and considerations for pretraining BitNet language models,...
Thank you for your excellent work on BitNet. Currently, it seems that iOS is not supported. Could you please consider adding support for iOS? Making BitNet available on iOS would...
Problem: convert-helper-bitnet.py throws an error on Windows. Can't find llama-quantize executable. Original path is configured for linux. Solution: Added .exe extension to the file path based on OS type
BitNet Distillation (https://arxiv.org/pdf/2510.13998) is interesting topic.
