BitNet
BitNet copied to clipboard
microsoft
Official inference framework for 1-bit LLMs
Dear Authors, Thanks for introducing the amazing project. When I tested the BitNet Inference Kernel on RTX 3090 with Ubuntu system, I followed the commands in README.md, but I got...
### Description This pull request addresses a grammatical error and improves the clarity of a sentence in the `README.md` file. ### Changes Made - In the "Benchmark" section, the sentence...
why not using light wight all in one python tools like UV? conda are too bloated.
When I try to download the model to compile it, it stays in this state: (miniconda-bitnet-p3_9) bitnet@BitNet:~/BitNet$ hf download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T Fetching 3 files: 0%| | 0/3 [00:00
Hey all! I am attempting to run and benchmark TL1 inference on my Mac M1 Pro device. After following the guide for the build process successfully, I ran `python setup_env.py...
== Running in interactive mode. == - Press Ctrl+C to interject at any time. - Press Return to return control to the AI. - To return control without starting a...
On Apple M2 silicon it seems that the BitNet code is triggering a LLVM optimization bug. Others have reported this as the build running for hours: issue #251, #260, etc....
I noticed that we’ve defined several custom functions for BitNet inference. However, I’m curious—how does llama.cpp know when and where to call these functions? Specifically, how are these BitNet-related functions...
