BitNet
BitNet copied to clipboard
microsoft
Official inference framework for 1-bit LLMs
Hi, I would like to know, the possibilities for integrating the speech encoder to turn this model input Speech + Text input instead of Text only input. Any information regarding...
Hi, given I want to translate a simple sentence python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "Translate me this sentence into french : the white rabbit jumps over the white rainbow" -n...
The old issue https://github.com/microsoft/BitNet/issues/75 was closed. So I had to post it again... I've below CLang in Windows 11. ``` \BitNet> clang --version clang version 19.1.7 Target: x86_64-w64-windows-gnu Thread model:...
As per title, adds support of Falcon-E models: https://huggingface.co/collections/tiiuae/falcon-edge-series-6804fd13344d6d8a8fa71130 Needs: https://github.com/Eddie-Wang1120/llama.cpp/pull/8 to be merged
Intel(R) Core(TM) i3-3217U CPU @ 1.80GHz === System: Windows10(PowerShell--conda) === Python==3.9.22 ```bash cmake --version cmake version 3.29.5-msvc4 clang --version clang version 18.1.8 Target: i686-pc-windows-msvc Thread model: posix InstalledDir: D:\Program Files...
I have multiple machines with many CPUs, but each machine's token is slow (2 t/s), is there any way to deploy BitNet distributely so that I can utilize all Idle...
Should be squashed
```txt (bitnet-cpp) PS C:\Users\hoofcushion\Projects\BitNet> python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s INFO:root:Compiling the code using CMake. ERROR:root:Error occurred while running command: Command '['cmake', '-B', 'build', '-DBITNET_X86_TL2=ON', '-T', 'ClangCL', '-DCMAKE_C_COMPILER=clang', '-DCMAKE_CXX_COMPILER=clang++']' returned...
Hello, I am following the root README, I've managed to compile the sources. When I run the inference command example `python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant"...
