BitNet
BitNet copied to clipboard
microsoft
Official inference framework for 1-bit LLMs
I'm wondering if the [microsoft/bitnet-b1.58-2B-4T-bf16](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-bf16) variant is the weights before SFT and DPO.
Dear All, I tested the code on a GeForce 2060 GPU and seems that the `W2A8` matmul output is wrong. The same code works properly with an RTX A5000 GPU...
Hi developers, When I followed the tutorial (Build from source), and run `python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s`, the printed error is: ``` INFO:root:Compiling the code using CMake. ERROR:root:Error occurred...
readme instruction: `huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T` > ⚠️ Warning: 'huggingface-cli download' is deprecated. Use 'hf download' instead.
I downloaded the ```1bitLLM/bitnet_b1_58-large``` and ```1bitLLM/bitnet_b1_58-3B``` models. When I run the inference commands, the model does not error out but produces incorrect outputs. ``` python setup_env.py -md models/bitnet_b1_58-3B -q tl1...
## Summary This PR introduces necessary changes to the build system, CMake files, and source code to enable successful compilation and running of BitNet using MinGW (Minimalist GNU for Windows)...
### The code used : https://colab.research.google.com/#fileId=https%3A//huggingface.co/microsoft/bitnet-b1.58-2B-4T.ipynb ## Issue : ``` --------------------------------------------------------------------------- OSError Traceback (most recent call last) [/tmp/ipython-input-4267558601.py](https://localhost:8080/#) in () 2 from transformers import pipeline 3 ----> 4 pipe =...
system: windows11pro shell: developer powershell for vs 18 generate_build_files.log: ``` -- OpenMP found -- Using llamafile -- Warning: ccache not found - consider installing it for faster compilation or disable...
when cmd shows that python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s INFO:root:Compiling the code using CMake. ERROR:root:Error occurred while running command: Command '['cmake', '--build', 'build', '--config', 'Release']' returned non-zero exit status...
Hi BitNet team, I've been exploring BitNet and am impressed with the performance improvements. I'd like to suggest adding comprehensive documentation on GPU inference best practices. **Suggested improvements:** - Example...
