BitNet
BitNet copied to clipboard
fix for Microsoft's incompetence
export CC=clang
export CXX=clang++
cmake -B build -DGGML_BITNET_ARM_TL1=ON -DCMAKE_C_COMPILER=clang-19 -DCMAKE_CXX_COMPILER=clang++-19
CC=/usr/bin/clang CXX=/usr/bin/clang++ python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s
one wonders how they got 15k stars when ikarakrow llamacpp fork is 9 months old and runs much better
comparison between this fork and ikawrakow, running on rk3588 SBC
llama_perf_context_print: prompt eval time = 9751.63 ms / 16 tokens ( 609.48 ms per token, 1.64 tokens per second)
llama_perf_context_print: eval time = 1379.77 ms / 9 runs ( 153.31 ms per token, 6.52 tokens per second)
llama_print_timings: prompt eval time = 90.99 ms / 6 tokens ( 15.17 ms per token, 65.94 tokens per second)
llama_print_timings: eval time = 4644.74 ms / 127 runs ( 36.57 ms per token, 27.34 tokens per second)